-- Logs begin at Thu 2022-09-08 18:02:13 UTC, end at Thu 2022-09-08 19:45:29 UTC. -- Sep 08 18:31:38.719787 dv-cinder-856463-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 08 18:31:38.729094 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 8 18:31:38 2022] *** Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: nodename: dv-cinder-856463-3-iscsi Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: machine: x86_64 Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: clock source: unix Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: pcre jit disabled Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: detected number of CPU cores: 4 Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: current working directory: / Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: detected binary path: /usr/bin/uwsgi-core Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: your processes number limit is 47680 Sep 08 18:31:38.731178 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: your memory page size is 4096 bytes Sep 08 18:31:38.731859 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: detected max file descriptor number: 2048 Sep 08 18:31:38.731859 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: lock engine: pthread robust mutexes Sep 08 18:31:38.731859 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: thunder lock: enabled Sep 08 18:31:38.731859 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 08 18:31:38.731859 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 08 18:31:38.769912 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: Python main interpreter initialized at 0x55940aa239c0 Sep 08 18:31:38.769912 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: python threads support enabled Sep 08 18:31:38.769912 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: your server socket listen backlog is limited to 100 connections Sep 08 18:31:38.769912 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: your mercy for graceful operations on workers is 90 seconds Sep 08 18:31:38.770272 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: mapped 403077 bytes (393 KB) for 2 cores Sep 08 18:31:38.770350 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: *** Operational MODE: preforking *** Sep 08 18:31:38.770591 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: *** uWSGI is running in multiple interpreter mode *** Sep 08 18:31:38.770591 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: spawned uWSGI master process (pid: 88159) Sep 08 18:31:38.770793 dv-cinder-856463-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 08 18:31:38.771920 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: spawned uWSGI worker 1 (pid: 88172, cores: 1) Sep 08 18:31:38.771920 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: spawned uWSGI worker 2 (pid: 88173, cores: 1) Sep 08 18:31:38.771920 dv-cinder-856463-3-iscsi devstack@keystone.service[88159]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 08 18:31:43.174814 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 18:31:43.174814 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: from cryptography.utils import int_from_bytes Sep 08 18:31:43.175602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 18:31:43.175602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: from cryptography.utils import int_from_bytes Sep 08 18:31:43.176880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 18:31:43.176880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: from cryptography.utils import int_from_bytes Sep 08 18:31:43.177388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 18:31:43.177388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: from cryptography.utils import int_from_bytes Sep 08 18:31:43.225172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 18:31:43.225341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 18:31:43.225535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] command line args: [] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 18:31:43.225806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 18:31:43.225982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ================================================================================ {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 18:31:43.226237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] admin_token = **** {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.226482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] config_dir = [] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.226670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.226854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] config_source = [] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.227035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] debug = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.227232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.228441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.228539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 18:31:43.228609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.228679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 18:31:43.228745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.228811 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] command line args: [] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 18:31:43.228875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.228875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] list_limit = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.228989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 18:31:43.229081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] log_config_append = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.229145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ================================================================================ {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 18:31:43.229216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.229281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] admin_token = **** {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.229345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] log_dir = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.229403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] log_file = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.229469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] config_dir = [] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.229666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.229742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.229809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] config_source = [] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.229879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.229990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] debug = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.230079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.230158 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.230308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.230396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.230522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.230611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.230684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.230802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.230884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.230988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.231114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.231233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] list_limit = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.231347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.231456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] log_config_append = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.231597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.231674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.231803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] log_dir = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.231872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.231949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] log_file = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.232101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.232195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.232308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.232405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.232554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.232638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.232825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.232907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.233030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.233202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.233340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.233452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.233539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] publish_errors = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.233704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.233792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.233942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.234018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.234135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.234221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.234360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.234445 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.234596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.234679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.234831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] standard_threads = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.234920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.235064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.235142 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.235307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.235391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.235484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.235660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] use_journal = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.235822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.235906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] use_json = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.235989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] use_stderr = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.236066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] publish_errors = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.236143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] use_syslog = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.236215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.236375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.236458 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.236612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.236695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.236862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.236982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.237124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.237211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.237365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] standard_threads = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.237444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.237600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.237676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.237842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.237925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.238005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.238076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.238189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] use_journal = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.238317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.238394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] use_json = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.238532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] use_stderr = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.238684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.238758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] use_syslog = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.238828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.238932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 18:31:43.239037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.239190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.239267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.239431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.239508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.239696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.239776 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.239945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.240068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.240198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.240289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.240436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.240513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.240687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.240772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.240925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.241003 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.241202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.241340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.241482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.241615 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.241741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.241866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.241996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.242134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.242266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.242399 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.242526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.242663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.242790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.242915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.242915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.243166 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.243785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.243916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.244041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.244169 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.244317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.244442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.244567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.244700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.244823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.244947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.245202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.245333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.245461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.245586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.245712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.245838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.245971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.246095 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.246234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.246359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.246483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.246614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.246746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.246876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.247000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.247130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.247130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.247377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.247512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.247636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.247761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.247989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.248136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.248276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.248406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.248540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.248669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.248805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.248934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.249084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.249208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.249333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.249408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.249524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.249644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.249778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.249862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.249940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.250963 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.251028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.251097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.251172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.251252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.251385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.251460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.251618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.251704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.251877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.251966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.252165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.252243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.252400 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.252477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.252639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.252724 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.252883 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.253108 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.253108 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.253345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.253469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.253602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.253730 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.253864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.253998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.254126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.254261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.254394 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.254518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.254648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.254773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.254902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.255035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.255162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.255295 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.255420 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.255559 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.255685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.255809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.255941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.256072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.256214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.256336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.256461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.256461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.256771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.256900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.257024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.257165 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.257293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.257431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.257566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.257692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.257816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.257942 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.258067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.258067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.258315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.258315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.258565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.258689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.258813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.259521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.259654 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.259802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.259929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.260055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.260180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.260311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.260438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.260575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.260753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.260880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.260880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.261143 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.261271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.261409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.261546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.261679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.261804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.261929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.262053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.262185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.262268 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.262435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.262561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.262687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.262811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.262943 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.263069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.263193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.263328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.263455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.263581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.263714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.263839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] auth.external = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.263962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.264086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.264209 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.264334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.264467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.264593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.264726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.264851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] auth.password = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.264974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.265124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] auth.token = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.265262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.265398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.265564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.265706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.265836 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.265959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.266087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.266215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.266346 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.266470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.266599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.266723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.266857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.266980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.267111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.267241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] credential.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.267370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.267494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.267616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] auth.external = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.267740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.267874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.268004 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.268141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.268273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.268396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.268527 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.268660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] auth.password = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.268785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.268915 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] auth.token = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.269038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.269209 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.269348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.269475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.269601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.269726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.269854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.269978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.269978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.270236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.270373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.270499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.270625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.270749 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.270889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] credential.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.271015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.271139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.271264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.271389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.271514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] federation.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.271646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.271771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.271896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.272021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.272155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.272281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.272281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.272586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.272712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.272837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.272965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.273111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.273245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.273375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.273504 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.273632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.273759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.273897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.274034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.274163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.274298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.274426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.274564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.274690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.274826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.275270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.275404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.275534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] federation.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.275661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.275789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.275929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.276071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.276196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.276320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.276443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.276566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.276696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.276821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.276945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.277086 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.277211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.277339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.277480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.277480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.277731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.277859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.277859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.278100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.278238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.278366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.278493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.278616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.278744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.278872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.279015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.279146 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.279275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.279402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.279540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.279673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.279802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.279929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.280057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.280184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.280313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.280448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.280581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.280709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.280836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.280959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.281135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.281283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.281413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.281540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.281666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.281791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.281916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.282048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.282181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.282314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.282442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.282566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.282693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.282827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.282953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.283088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.283217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.283342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.283470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.283603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.283729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.283855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.283981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.284106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.284231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.284363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.284491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.284617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.284743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.284878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.285004 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.285182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.285309 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.285432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.285557 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.285679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.285808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.285945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.286074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.286201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.286330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.286457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.286591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.286723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.286851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.286981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.287104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.287232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.287362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.287493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.287618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.287741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.287863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.287986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.288114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.288245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.288384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.288513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.288642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.288770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.288893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.289024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.289185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.289312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.289440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.289565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.289691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.289828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.289956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.290082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.290212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.290212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.290459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.290598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.290728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.290857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.290982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.291106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.291229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.291364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.291493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.291620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.291747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.291888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.292016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.292151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.292282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.292408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.292490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.292552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.292724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.292850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.292976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293270 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293918 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.293976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294042 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.294987 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] resource.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.295045 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.295531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.295667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.295794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.295918 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.296043 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.296167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.296305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.298114 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.298249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.298380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.298507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.298633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.298735 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.298801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.298861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.298919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.298977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] role.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] role.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.299955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] resource.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.300969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.301035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.301450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.301450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.301695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.301822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.301961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.302097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] role.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.302226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.302360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] role.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.302482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.302564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.302740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.302869 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.303516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.303647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.303777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.303905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.304170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.304331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.304470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.304600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.304727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.304852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.304988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.305133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.305257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.305384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.305511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.305636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.306332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.306543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.306689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.306816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.306941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.307065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.307199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.307316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.307440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.307440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.307681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.307805 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] token.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.307937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.308061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.308184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.308310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.308444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.308571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.308707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.308832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.309241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.309370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.309497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.309623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.309623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.309869 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.309994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.310117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.310242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.310375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.310506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.310630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.310756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.310880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.311005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.311005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.311244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.311369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.311491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.311616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.311749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.311879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.312006 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.312146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.312271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.312394 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.connection = **** {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.312517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.312517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.312748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.312882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] token.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.312882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.313130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.313257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.313257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.313491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.313626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.313760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.313887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.313887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.314119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.314241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.314241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.314481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.314609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.314733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.314859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.314859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.315101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.315225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.315225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.315465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.315589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.315714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.315845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.315845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.316096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.316221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.316345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.316469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.connection = **** {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.316469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.316780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.316908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.317035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.317190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.317190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.317438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.317574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.317699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.317824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.317824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318439 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318439 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 18:31:43.318797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 18:31:43.318797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 18:31:43.348768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.348990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.349236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.356188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.356421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.356648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.358027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.364948 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.387767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.387996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.391879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.392113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.442461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.442751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.492169 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.492397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.492535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.492616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.651979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.652192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.652282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.652417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.652583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.652666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.652847 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.652968 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.653071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.653320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.653608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.653893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.676471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.676732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.676816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.676960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.677200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.677299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.677425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.677596 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.677678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.677847 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.677962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.678250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.678529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.678809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.689774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.690111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.702654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.702829 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.703062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.703194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.703318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.703465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.703601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.703724 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.703850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.703980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.704181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.704469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.704752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.705129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 18:31:43.720736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.720893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.720981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 18:31:43.721224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 18:31:43.721313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.721496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 18:31:43.721733 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.721831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.722020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.722102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 18:31:43.722278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.722491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.722574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.722864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.723189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.723509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.732598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.732879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.733177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.733462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.734392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.734744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.735084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.735437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.740397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.742645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.743859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.744150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.744432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.744716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.744997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.745285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.745555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.745834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.746102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.746374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.746640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.747215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.747571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.747921 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.748287 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.748643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.748974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.749338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.749679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.750030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.750370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.750709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.769466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.770938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.771233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.774022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.775997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.776154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.776363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.776469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.776706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.781066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.781358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.781645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.783145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.783435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.787973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.788265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.789032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.789352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.789637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.789916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.790709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.790978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.791245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.791513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.794823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.795124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.795417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.795709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.795993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.796276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.796553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.796830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.806931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.807209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.811032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.812596 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.812880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.814214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.814510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.816696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.818654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.819901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.820203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.822017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.824863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.825261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.825565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.825851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.826134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.826410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.826698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.827962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.831177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.831481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.831774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.832065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.832346 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.832626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.843178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.843465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.843732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.843994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.849370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.849664 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.849938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.850205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.850816 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.852355 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.854099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.854377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.857141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.857429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.857510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.857713 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.857993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.858385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.859005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.860543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.860829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.863621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.863912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.864198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.864482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.864875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.870189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.870475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.870754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.871034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.871414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.871697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.871985 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.876988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.877299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.877580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.878155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.878249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.894109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.894109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.894259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.894259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.894259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.894654 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.894740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.896168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.896453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.896857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.897589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.901457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.902944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.903852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.904412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.904709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.905139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.905321 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.908196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.908590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.908874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.909187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.911557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.913025 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.916322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.916726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.916993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.917099 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.917308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.917390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.917589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.924434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.924717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.924997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.925194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.925306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.925502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.925586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.925835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.925922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.926107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.926382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.926648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.926928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.927198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.927469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.927743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.928007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.933689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 18:31:43.933975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.934261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.934545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.934827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.935102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.935377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.935663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.935935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.936226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.936549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.936824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.937166 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.937484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 18:31:43.962864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55940aa239c0 pid: 88173 (default app) Sep 08 18:31:43.966192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de96368-7ebc-409b-a5ef-6251c7482426 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:43.966735 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de96368-7ebc-409b-a5ef-6251c7482426 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:43.967008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de96368-7ebc-409b-a5ef-6251c7482426 None None] PATH_INFO: `/v3` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:43.971181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Thu Sep 8 18:31:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:31:43.973292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55940aa239c0 pid: 88172 (default app) Sep 08 18:31:51.522845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87690b86-aba2-4f1d-91f0-1e46656e6a30 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:51.523367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87690b86-aba2-4f1d-91f0-1e46656e6a30 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:51.523367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87690b86-aba2-4f1d-91f0-1e46656e6a30 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:51.524697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 2/2] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:31:51.533127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:51.533577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:51.533677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:51.605032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: INFO dbcounter [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] Registered counter for database keystone Sep 08 18:31:51.623801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG oslo_db.sqlalchemy.engines [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88172) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 18:31:51.629955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writer thread running {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 18:31:51.810171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG passlib.handlers.bcrypt [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88172) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 18:31:51.810565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG passlib.handlers.bcrypt [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88172) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 18:31:51.867875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:31:51.870029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:31:51.997053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1/3] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:31:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 08 18:31:52.444225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87690b86-aba2-4f1d-91f0-1e46656e6a30 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:52.514786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: INFO dbcounter [None req-87690b86-aba2-4f1d-91f0-1e46656e6a30 None None] Registered counter for database keystone Sep 08 18:31:52.536285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG oslo_db.sqlalchemy.engines [None req-87690b86-aba2-4f1d-91f0-1e46656e6a30 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88173) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 18:31:52.542570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writer thread running {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 18:31:52.662226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87690b86-aba2-4f1d-91f0-1e46656e6a30 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:52.667043 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad973a26-9b42-40e3-a3dc-1b3eabb58c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:52.667670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad973a26-9b42-40e3-a3dc-1b3eabb58c53 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:52.667950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad973a26-9b42-40e3-a3dc-1b3eabb58c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:52.668223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad973a26-9b42-40e3-a3dc-1b3eabb58c53 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:52.676456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad973a26-9b42-40e3-a3dc-1b3eabb58c53 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:52.730011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ad973a26-9b42-40e3-a3dc-1b3eabb58c53 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 18:31:52.730285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad973a26-9b42-40e3-a3dc-1b3eabb58c53 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:52.740272 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 3/4] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Sep 8 18:31:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 18:31:52.746867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:52.763599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829d2bb-41ac-407c-84d8-b2d8c7911812 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:52.768071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e38f6-378c-4df1-808a-36261550f019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:52.769367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e38f6-378c-4df1-808a-36261550f019 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:52.769661 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e38f6-378c-4df1-808a-36261550f019 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:52.769948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e38f6-378c-4df1-808a-36261550f019 admin admin] PATH_INFO: `/v3/projects/1a0304ec85d64b64aa7cbc3f8a24053a` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:52.771399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e38f6-378c-4df1-808a-36261550f019 admin admin] RBAC: Authorizing `identity:get_project(project_id=1a0304ec85d64b64aa7cbc3f8a24053a)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:52.824232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: WARNING keystone.common.rbac_enforcer.enforcer [None req-480e38f6-378c-4df1-808a-36261550f019 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 18:31:52.824577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e38f6-378c-4df1-808a-36261550f019 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:52.829132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 2/5] 10.222.0.25 () {56 vars in 1207 bytes} [Thu Sep 8 18:31:52 2022] GET /identity/v3/projects/1a0304ec85d64b64aa7cbc3f8a24053a => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:31:55.000576 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c012-6eed-4785-a490-97010562c525 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:55.001114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c012-6eed-4785-a490-97010562c525 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:55.001114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c012-6eed-4785-a490-97010562c525 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:55.005191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 4/6] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:31:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:31:55.011197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152559bd-9dc6-4a8d-835f-bf2a7a4dd3ca None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:55.011607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152559bd-9dc6-4a8d-835f-bf2a7a4dd3ca None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:55.011757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152559bd-9dc6-4a8d-835f-bf2a7a4dd3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:55.032007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-152559bd-9dc6-4a8d-835f-bf2a7a4dd3ca None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:31:55.038444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-152559bd-9dc6-4a8d-835f-bf2a7a4dd3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:31:55.041517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 3/7] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:31:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 08 18:31:55.307257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8c012-6eed-4785-a490-97010562c525 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:55.319318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8c012-6eed-4785-a490-97010562c525 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:55.324198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b1936-439e-4437-bce5-a73df7cfd0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:55.324792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b1936-439e-4437-bce5-a73df7cfd0dd admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:55.325093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b1936-439e-4437-bce5-a73df7cfd0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:55.325370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b1936-439e-4437-bce5-a73df7cfd0dd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:55.332837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b1936-439e-4437-bce5-a73df7cfd0dd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:55.334628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b1936-439e-4437-bce5-a73df7cfd0dd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:55.343531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 5/8] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Sep 8 18:31:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 18:31:55.348708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152559bd-9dc6-4a8d-835f-bf2a7a4dd3ca None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:55.357971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152559bd-9dc6-4a8d-835f-bf2a7a4dd3ca None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:55.362272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7946f8-f619-4c78-af5a-fff3025665bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:55.362900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7946f8-f619-4c78-af5a-fff3025665bc admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:55.363181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7946f8-f619-4c78-af5a-fff3025665bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:55.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7946f8-f619-4c78-af5a-fff3025665bc admin admin] PATH_INFO: `/v3/users/ba1856b68bd244e69db7d867dbb38da3` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:55.364760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7946f8-f619-4c78-af5a-fff3025665bc admin admin] RBAC: Authorizing `identity:get_user(user_id=ba1856b68bd244e69db7d867dbb38da3)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:55.366618 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7946f8-f619-4c78-af5a-fff3025665bc admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:55.368669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 4/9] 10.222.0.25 () {56 vars in 1198 bytes} [Thu Sep 8 18:31:55 2022] GET /identity/v3/users/ba1856b68bd244e69db7d867dbb38da3 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:31:57.602255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a0604-fc57-4a61-b92d-20e9c09aef82 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:57.603429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a0604-fc57-4a61-b92d-20e9c09aef82 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:57.603429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a0604-fc57-4a61-b92d-20e9c09aef82 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:57.604910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 6/10] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:31:57.610387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba41b3-3fcf-4d98-8483-b1c4a0818710 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:57.610809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba41b3-3fcf-4d98-8483-b1c4a0818710 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:57.610943 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba41b3-3fcf-4d98-8483-b1c4a0818710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:57.630186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-7aba41b3-3fcf-4d98-8483-b1c4a0818710 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:31:57.631525 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7aba41b3-3fcf-4d98-8483-b1c4a0818710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:31:57.664322 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca14be2-7c72-455f-99c8-870878479281 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:57.664609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca14be2-7c72-455f-99c8-870878479281 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:57.664911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca14be2-7c72-455f-99c8-870878479281 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:57.666863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 7/11] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:31:57.671994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2d64d-4476-4f15-810e-f563bd9bef59 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:57.672277 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2d64d-4476-4f15-810e-f563bd9bef59 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:57.672551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2d64d-4476-4f15-810e-f563bd9bef59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:57.715266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG passlib.handlers.bcrypt [None req-c9e2d64d-4476-4f15-810e-f563bd9bef59 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88173) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 18:31:57.715650 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG passlib.handlers.bcrypt [None req-c9e2d64d-4476-4f15-810e-f563bd9bef59 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88173) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 18:31:57.717789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 5/12] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:31:57.769415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c9e2d64d-4476-4f15-810e-f563bd9bef59 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:31:57.771036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c9e2d64d-4476-4f15-810e-f563bd9bef59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:31:57.870578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 8/13] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:31:57.995546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba41b3-3fcf-4d98-8483-b1c4a0818710 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:58.021888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:31:58.103897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba41b3-3fcf-4d98-8483-b1c4a0818710 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:58.138573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc299d-def6-4d0d-9142-981c74588c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:58.139240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc299d-def6-4d0d-9142-981c74588c98 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:58.139532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc299d-def6-4d0d-9142-981c74588c98 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:58.139805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc299d-def6-4d0d-9142-981c74588c98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:58.140331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2d64d-4476-4f15-810e-f563bd9bef59 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:58.151937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc299d-def6-4d0d-9142-981c74588c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-a5bc299d-def6-4d0d-9142-981c74588c98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:31:58.160816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:31:58.161892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:31:58.162672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 6/14] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:31:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:31:58.168159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc299d-def6-4d0d-9142-981c74588c98 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:58.245584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2d64d-4476-4f15-810e-f563bd9bef59 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:58.247868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:31:58.250971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:31:58.268967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc299d-def6-4d0d-9142-981c74588c98 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:58.283682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc980c-7e1d-4ed4-90d0-c39f6ce799cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:58.284557 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc980c-7e1d-4ed4-90d0-c39f6ce799cb None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:58.284841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc980c-7e1d-4ed4-90d0-c39f6ce799cb None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:58.285141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc980c-7e1d-4ed4-90d0-c39f6ce799cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:58.304172 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b644ec9-0859-49fd-aa66-cbbb9fed6419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:58.304774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b644ec9-0859-49fd-aa66-cbbb9fed6419 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:58.305070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b644ec9-0859-49fd-aa66-cbbb9fed6419 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:58.305354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b644ec9-0859-49fd-aa66-cbbb9fed6419 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:58.306562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b644ec9-0859-49fd-aa66-cbbb9fed6419 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:31:58.306859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b644ec9-0859-49fd-aa66-cbbb9fed6419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:58.308257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b644ec9-0859-49fd-aa66-cbbb9fed6419 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:58.320090 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 7/15] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:31:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:31:58.325807 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b644ec9-0859-49fd-aa66-cbbb9fed6419 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:58.354623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc980c-7e1d-4ed4-90d0-c39f6ce799cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:58.356137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc980c-7e1d-4ed4-90d0-c39f6ce799cb None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:58.426711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b644ec9-0859-49fd-aa66-cbbb9fed6419 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:58.455137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 9/16] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:31:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 18:31:58.460691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc980c-7e1d-4ed4-90d0-c39f6ce799cb None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:58.460900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaffb2c-c19b-46ba-87c7-95c1d3b6658b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:58.461555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaffb2c-c19b-46ba-87c7-95c1d3b6658b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:58.461929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaffb2c-c19b-46ba-87c7-95c1d3b6658b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:58.462207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaffb2c-c19b-46ba-87c7-95c1d3b6658b None admin] PATH_INFO: `/v3/users/ba1856b68bd244e69db7d867dbb38da3` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:58.463351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaffb2c-c19b-46ba-87c7-95c1d3b6658b None admin] RBAC: Authorizing `identity:get_user(user_id=ba1856b68bd244e69db7d867dbb38da3)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:58.463765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:31:58.464809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaffb2c-c19b-46ba-87c7-95c1d3b6658b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:58.466422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 8/17] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:31:58 2022] GET /identity/v3/users/ba1856b68bd244e69db7d867dbb38da3 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:31:58.473194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaffb2c-c19b-46ba-87c7-95c1d3b6658b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:58.488353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:31:58.559644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc980c-7e1d-4ed4-90d0-c39f6ce799cb None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:58.573924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaffb2c-c19b-46ba-87c7-95c1d3b6658b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:58.595329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be180f3-8917-44df-8444-817d47995b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:58.596203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be180f3-8917-44df-8444-817d47995b2b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:58.596490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be180f3-8917-44df-8444-817d47995b2b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:58.596765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be180f3-8917-44df-8444-817d47995b2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:58.609400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de3fed-0669-494f-9db5-168cae6d2b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:58.610040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de3fed-0669-494f-9db5-168cae6d2b89 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:58.610321 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de3fed-0669-494f-9db5-168cae6d2b89 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:58.610591 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de3fed-0669-494f-9db5-168cae6d2b89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:58.611712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de3fed-0669-494f-9db5-168cae6d2b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:58.613197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de3fed-0669-494f-9db5-168cae6d2b89 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:58.614742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 9/18] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:31:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:31:58.616158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be180f3-8917-44df-8444-817d47995b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:58.617671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be180f3-8917-44df-8444-817d47995b2b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:58.621145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de3fed-0669-494f-9db5-168cae6d2b89 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-5be180f3-8917-44df-8444-817d47995b2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:31:58.641018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:31:58.642123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:31:58.643109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:31:58.643109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:31:58.643109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:31:58.643109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 10/19] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:31:58.649477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be180f3-8917-44df-8444-817d47995b2b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:31:58.707449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:31:58.712313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:31:58.732247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de3fed-0669-494f-9db5-168cae6d2b89 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:58.754112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be180f3-8917-44df-8444-817d47995b2b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:31:58.767425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ad7aa-c79a-461e-bb2b-b3b716d38791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:58.768035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad7aa-c79a-461e-bb2b-b3b716d38791 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:58.768317 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad7aa-c79a-461e-bb2b-b3b716d38791 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:58.768587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad7aa-c79a-461e-bb2b-b3b716d38791 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:58.769328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ad7aa-c79a-461e-bb2b-b3b716d38791 None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.domain.id=default, user.id=ba1856b68bd244e69db7d867dbb38da3) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:31:58.769633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ad7aa-c79a-461e-bb2b-b3b716d38791 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:58.771126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ad7aa-c79a-461e-bb2b-b3b716d38791 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:58.777989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 10/20] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Sep 8 18:31:58 2022] GET /identity/v3/role_assignments?user.id=ba1856b68bd244e69db7d867dbb38da3&scope.domain.id=default&role.id=5c777de40b8449329d714d8c2229115e => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:31:58.788240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ea859-7756-4c1d-a44f-3452cca5315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:31:58.788971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ea859-7756-4c1d-a44f-3452cca5315a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:31:58.789232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ea859-7756-4c1d-a44f-3452cca5315a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:31:58.789509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ea859-7756-4c1d-a44f-3452cca5315a None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:31:58.790927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ea859-7756-4c1d-a44f-3452cca5315a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:31:58.791202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ea859-7756-4c1d-a44f-3452cca5315a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:31:58.792663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ea859-7756-4c1d-a44f-3452cca5315a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:31:58.813017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 11/21] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:01.142818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0faba9-e4f2-48ee-9a17-4e352517cfec None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:01.143867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0faba9-e4f2-48ee-9a17-4e352517cfec None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:01.144412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0faba9-e4f2-48ee-9a17-4e352517cfec None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:01.146010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 11/22] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:01.152472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c6ab8-6e87-469c-b86a-9f98b7cd3a94 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:01.152472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c6ab8-6e87-469c-b86a-9f98b7cd3a94 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:01.152966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c6ab8-6e87-469c-b86a-9f98b7cd3a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:01.178484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d30c6ab8-6e87-469c-b86a-9f98b7cd3a94 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:01.179712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d30c6ab8-6e87-469c-b86a-9f98b7cd3a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:01.278583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 12/23] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:01.609419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e76b16-97ab-4a5a-881e-858e597a0faa None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:01.609951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e76b16-97ab-4a5a-881e-858e597a0faa None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:01.610431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e76b16-97ab-4a5a-881e-858e597a0faa None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:01.611976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 12/24] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:01.617412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea0a7a-5bfd-42ea-a56e-52f2d9e91148 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:01.617702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea0a7a-5bfd-42ea-a56e-52f2d9e91148 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:01.618048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea0a7a-5bfd-42ea-a56e-52f2d9e91148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:01.646938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-05ea0a7a-5bfd-42ea-a56e-52f2d9e91148 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:01.648146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-05ea0a7a-5bfd-42ea-a56e-52f2d9e91148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:01.707685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:01.716488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee034c3-eafe-47a6-b3f3-039f0d5a2ff0 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:01.717068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee034c3-eafe-47a6-b3f3-039f0d5a2ff0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:01.717341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee034c3-eafe-47a6-b3f3-039f0d5a2ff0 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:01.718900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 13/25] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:32:01.723798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33657b3-ef2a-4564-9f80-e28e38d80d07 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:01.729714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33657b3-ef2a-4564-9f80-e28e38d80d07 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:01.729714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33657b3-ef2a-4564-9f80-e28e38d80d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:01.755346 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 13/26] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:01.757027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f33657b3-ef2a-4564-9f80-e28e38d80d07 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:01.758301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f33657b3-ef2a-4564-9f80-e28e38d80d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:01.771318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea0a7a-5bfd-42ea-a56e-52f2d9e91148 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:01.922657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 14/27] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:32:01.947291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea0a7a-5bfd-42ea-a56e-52f2d9e91148 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:01.995361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a73cc-e13f-4a95-84e3-d2ebfc708242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:01.996062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a73cc-e13f-4a95-84e3-d2ebfc708242 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:01.996359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a73cc-e13f-4a95-84e3-d2ebfc708242 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:01.996636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a73cc-e13f-4a95-84e3-d2ebfc708242 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:02.008672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a73cc-e13f-4a95-84e3-d2ebfc708242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-1f0a73cc-e13f-4a95-84e3-d2ebfc708242 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:02.015290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:02.016181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:02.016967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 14/28] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:32:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:32:02.024800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33657b3-ef2a-4564-9f80-e28e38d80d07 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:02.029235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc281998-2a77-4bbc-8873-cf8fe95945d1 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:02.029520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc281998-2a77-4bbc-8873-cf8fe95945d1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:02.029849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc281998-2a77-4bbc-8873-cf8fe95945d1 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:02.031252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 15/29] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:32:02.042512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d4926-2fb9-48cd-b4af-ea1f45b9ef26 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:02.042798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d4926-2fb9-48cd-b4af-ea1f45b9ef26 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:02.043076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d4926-2fb9-48cd-b4af-ea1f45b9ef26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:02.077447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e07d4926-2fb9-48cd-b4af-ea1f45b9ef26 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:02.078840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e07d4926-2fb9-48cd-b4af-ea1f45b9ef26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:02.130958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:02.201422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:02.235462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 16/30] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:32:02.250732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d4926-2fb9-48cd-b4af-ea1f45b9ef26 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:02.261838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33657b3-ef2a-4564-9f80-e28e38d80d07 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:02.344821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f3843-234d-4e01-8e0c-5d0b25aa0a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:02.345496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f3843-234d-4e01-8e0c-5d0b25aa0a86 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:02.345778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f3843-234d-4e01-8e0c-5d0b25aa0a86 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:02.346050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f3843-234d-4e01-8e0c-5d0b25aa0a86 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:02.346730 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f3843-234d-4e01-8e0c-5d0b25aa0a86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:02.347034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f3843-234d-4e01-8e0c-5d0b25aa0a86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:02.348430 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f3843-234d-4e01-8e0c-5d0b25aa0a86 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:02.363035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 15/31] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:02.366619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f3843-234d-4e01-8e0c-5d0b25aa0a86 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:02.444131 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d4926-2fb9-48cd-b4af-ea1f45b9ef26 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:02.482254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d6091-1b35-419f-a3d5-e51fc973adff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:02.482815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d6091-1b35-419f-a3d5-e51fc973adff None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:02.483567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d6091-1b35-419f-a3d5-e51fc973adff None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:02.483849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d6091-1b35-419f-a3d5-e51fc973adff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:02.494515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d6091-1b35-419f-a3d5-e51fc973adff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:02.496164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d6091-1b35-419f-a3d5-e51fc973adff None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-ef3d6091-1b35-419f-a3d5-e51fc973adff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:32:02.511960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:02.512845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:32:02.513705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:32:02.513705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:32:02.513705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:02.513705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 17/32] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:32:02.513926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d6091-1b35-419f-a3d5-e51fc973adff None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:02.555193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:02.618682 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f3843-234d-4e01-8e0c-5d0b25aa0a86 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:02.664728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42685508-bdd5-4d26-9bc1-1463d99a10b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:02.665500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d6091-1b35-419f-a3d5-e51fc973adff None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:02.666045 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42685508-bdd5-4d26-9bc1-1463d99a10b2 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:02.666533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42685508-bdd5-4d26-9bc1-1463d99a10b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:02.667307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42685508-bdd5-4d26-9bc1-1463d99a10b2 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:02.669109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42685508-bdd5-4d26-9bc1-1463d99a10b2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:02.670734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42685508-bdd5-4d26-9bc1-1463d99a10b2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:02.679532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:02.703457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70615-3d30-4569-a442-565337dd18b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:02.704184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70615-3d30-4569-a442-565337dd18b7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:02.704532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70615-3d30-4569-a442-565337dd18b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:02.704908 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70615-3d30-4569-a442-565337dd18b7 None admin] PATH_INFO: `/v3/users/ba1856b68bd244e69db7d867dbb38da3` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:02.706488 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d70615-3d30-4569-a442-565337dd18b7 None admin] RBAC: Authorizing `identity:get_user(user_id=ba1856b68bd244e69db7d867dbb38da3)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:02.708082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d70615-3d30-4569-a442-565337dd18b7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:02.709876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 18/33] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:02 2022] GET /identity/v3/users/ba1856b68bd244e69db7d867dbb38da3 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:02.714909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70615-3d30-4569-a442-565337dd18b7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:02.821566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:02.838416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70615-3d30-4569-a442-565337dd18b7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:02.876883 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70576c88-fea2-417a-8a89-312b3666e46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:02.877552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70576c88-fea2-417a-8a89-312b3666e46f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:02.877835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70576c88-fea2-417a-8a89-312b3666e46f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:02.878108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70576c88-fea2-417a-8a89-312b3666e46f None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:02.878748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70576c88-fea2-417a-8a89-312b3666e46f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:02.879047 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70576c88-fea2-417a-8a89-312b3666e46f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:02.880539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70576c88-fea2-417a-8a89-312b3666e46f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:02.893408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 19/34] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:02.898249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70576c88-fea2-417a-8a89-312b3666e46f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:02.914892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 16/35] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 8 18:32:02 2022] POST /identity/v3/roles => generated 219 bytes in 551 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:02.916532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42685508-bdd5-4d26-9bc1-1463d99a10b2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:03.045778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70576c88-fea2-417a-8a89-312b3666e46f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:03.067515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42685508-bdd5-4d26-9bc1-1463d99a10b2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:03.090584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be35ad-c7d8-4291-8b4f-3bf56b6e3b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:03.091121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be35ad-c7d8-4291-8b4f-3bf56b6e3b66 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:03.091402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be35ad-c7d8-4291-8b4f-3bf56b6e3b66 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:03.091664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be35ad-c7d8-4291-8b4f-3bf56b6e3b66 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:03.092692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be35ad-c7d8-4291-8b4f-3bf56b6e3b66 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:03.094848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be35ad-c7d8-4291-8b4f-3bf56b6e3b66 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:03.115146 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cb422-7c35-443c-9138-c1347f56ba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:03.115802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cb422-7c35-443c-9138-c1347f56ba9c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:03.116096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cb422-7c35-443c-9138-c1347f56ba9c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:03.119268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cb422-7c35-443c-9138-c1347f56ba9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:03.119268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cb422-7c35-443c-9138-c1347f56ba9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:03.120315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cb422-7c35-443c-9138-c1347f56ba9c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:03.122243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 17/36] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:03.122846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cb422-7c35-443c-9138-c1347f56ba9c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:03.197505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:03.236312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cb422-7c35-443c-9138-c1347f56ba9c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:03.270833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf4463-23d6-44e2-b1dd-09fc4a2bcf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:03.271449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf4463-23d6-44e2-b1dd-09fc4a2bcf97 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:03.271738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf4463-23d6-44e2-b1dd-09fc4a2bcf97 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:03.272021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf4463-23d6-44e2-b1dd-09fc4a2bcf97 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:03.272683 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf4463-23d6-44e2-b1dd-09fc4a2bcf97 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:03.274835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf4463-23d6-44e2-b1dd-09fc4a2bcf97 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:03.283879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 20/37] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 8 18:32:02 2022] POST /identity/v3/roles => generated 225 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:03.285367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba42b3f-9c38-4ec6-9cb8-0efe6393038d None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:03.285659 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba42b3f-9c38-4ec6-9cb8-0efe6393038d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:03.285939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba42b3f-9c38-4ec6-9cb8-0efe6393038d None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:03.292243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 21/38] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:03.292243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba42b3f-9c38-4ec6-9cb8-0efe6393038d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:03.300112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:03.407736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba42b3f-9c38-4ec6-9cb8-0efe6393038d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:03.421625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 18/39] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:32:03 2022] POST /identity/v3/projects => generated 301 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:03.423409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de8fa4-183a-4bd7-9046-10c1322dcf46 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:03.423889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de8fa4-183a-4bd7-9046-10c1322dcf46 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:03.424178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de8fa4-183a-4bd7-9046-10c1322dcf46 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:03.426208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 19/40] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:03.427195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2938b-4805-47b8-83ae-d8d16c1e6cd5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:03.427557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2938b-4805-47b8-83ae-d8d16c1e6cd5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:03.427844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2938b-4805-47b8-83ae-d8d16c1e6cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:03.452367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-d8f2938b-4805-47b8-83ae-d8d16c1e6cd5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:03.454120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d8f2938b-4805-47b8-83ae-d8d16c1e6cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:03.468913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12246e74-04e8-4d20-928c-e9ba18c61833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:03.470015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12246e74-04e8-4d20-928c-e9ba18c61833 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:03.470518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12246e74-04e8-4d20-928c-e9ba18c61833 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:03.470898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12246e74-04e8-4d20-928c-e9ba18c61833 None admin] PATH_INFO: `/v3/domains/default/users/ba1856b68bd244e69db7d867dbb38da3/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:03.473314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12246e74-04e8-4d20-928c-e9ba18c61833 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ba1856b68bd244e69db7d867dbb38da3, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:03.475324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12246e74-04e8-4d20-928c-e9ba18c61833 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:03.583923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 22/41] 10.222.0.25 () {58 vars in 1361 bytes} [Thu Sep 8 18:32:03 2022] PUT /identity/v3/domains/default/users/ba1856b68bd244e69db7d867dbb38da3/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:32:03.585336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c375ef-1356-44c2-b4d0-37788d9bdb7d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:03.586007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c375ef-1356-44c2-b4d0-37788d9bdb7d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:03.586007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c375ef-1356-44c2-b4d0-37788d9bdb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:03.630271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-02c375ef-1356-44c2-b4d0-37788d9bdb7d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:03.631995 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-02c375ef-1356-44c2-b4d0-37788d9bdb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:03.684219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 20/42] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:03.716983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:03.723051 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 23/43] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:03.966035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2938b-4805-47b8-83ae-d8d16c1e6cd5 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:04.002602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c375ef-1356-44c2-b4d0-37788d9bdb7d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:04.019046 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:04.077260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2938b-4805-47b8-83ae-d8d16c1e6cd5 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:04.112805 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c375ef-1356-44c2-b4d0-37788d9bdb7d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:04.119580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca682f-97e1-4be4-8e4f-642d95df6a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:04.120437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca682f-97e1-4be4-8e4f-642d95df6a40 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:04.121473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca682f-97e1-4be4-8e4f-642d95df6a40 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:04.121951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca682f-97e1-4be4-8e4f-642d95df6a40 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:04.122887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca682f-97e1-4be4-8e4f-642d95df6a40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:04.125268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca682f-97e1-4be4-8e4f-642d95df6a40 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:04.150140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de209aa-5579-4616-a46d-4ddeb3ea2717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:04.150784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de209aa-5579-4616-a46d-4ddeb3ea2717 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:04.151069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de209aa-5579-4616-a46d-4ddeb3ea2717 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:04.151348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de209aa-5579-4616-a46d-4ddeb3ea2717 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:04.152469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de209aa-5579-4616-a46d-4ddeb3ea2717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:04.153957 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de209aa-5579-4616-a46d-4ddeb3ea2717 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:04.155763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 24/44] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:04.160939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de209aa-5579-4616-a46d-4ddeb3ea2717 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:04.196635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 21/45] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 8 18:32:03 2022] POST /identity/v3/roles => generated 223 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:04.228792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:04.273263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de209aa-5579-4616-a46d-4ddeb3ea2717 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:04.311442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98fed7-9961-49e3-941e-d889c7da25b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:04.312093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98fed7-9961-49e3-941e-d889c7da25b0 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:04.312358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98fed7-9961-49e3-941e-d889c7da25b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:04.312633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98fed7-9961-49e3-941e-d889c7da25b0 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:04.313338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98fed7-9961-49e3-941e-d889c7da25b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:04.314819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98fed7-9961-49e3-941e-d889c7da25b0 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:04.391674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 25/46] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:32:04 2022] POST /identity/v3/projects => generated 312 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:32:05.798815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599deeb8-0f42-47f8-a62a-5de733775896 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:05.799320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599deeb8-0f42-47f8-a62a-5de733775896 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:05.799421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599deeb8-0f42-47f8-a62a-5de733775896 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:05.800747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 22/47] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:05.806597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aad2c-ed14-49ce-8540-aee90b7f03da None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:05.806998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aad2c-ed14-49ce-8540-aee90b7f03da None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:05.807146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727aad2c-ed14-49ce-8540-aee90b7f03da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:05.829963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-727aad2c-ed14-49ce-8540-aee90b7f03da None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:05.831142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-727aad2c-ed14-49ce-8540-aee90b7f03da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:05.898175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 26/48] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:06.167809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599deeb8-0f42-47f8-a62a-5de733775896 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:06.270025 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599deeb8-0f42-47f8-a62a-5de733775896 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:06.277324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:06.303881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508c8c1-76d5-4d1e-aa99-f7c2ba2948d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:06.304511 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508c8c1-76d5-4d1e-aa99-f7c2ba2948d7 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:06.304792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508c8c1-76d5-4d1e-aa99-f7c2ba2948d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:06.305089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508c8c1-76d5-4d1e-aa99-f7c2ba2948d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:06.316172 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508c8c1-76d5-4d1e-aa99-f7c2ba2948d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-c508c8c1-76d5-4d1e-aa99-f7c2ba2948d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:06.321528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:06.322641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:06.323398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 23/49] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:32:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:06.328500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727aad2c-ed14-49ce-8540-aee90b7f03da None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:06.481189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:06.534766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727aad2c-ed14-49ce-8540-aee90b7f03da None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:06.544955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402965de-ee21-4007-8417-b5eb08e755c9 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:06.545356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402965de-ee21-4007-8417-b5eb08e755c9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:06.545703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402965de-ee21-4007-8417-b5eb08e755c9 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:06.547126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 24/50] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:32:06.552224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ea2cf-8ee4-4d2d-ba1e-fb4a06f70981 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:06.552598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ea2cf-8ee4-4d2d-ba1e-fb4a06f70981 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:06.552876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ea2cf-8ee4-4d2d-ba1e-fb4a06f70981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:06.569387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d01a2c-51b0-4ad9-9cf5-7641377b49c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:06.570014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d01a2c-51b0-4ad9-9cf5-7641377b49c7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:06.570301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d01a2c-51b0-4ad9-9cf5-7641377b49c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:06.570583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d01a2c-51b0-4ad9-9cf5-7641377b49c7 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:06.571958 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a39ea2cf-8ee4-4d2d-ba1e-fb4a06f70981 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:06.572079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d01a2c-51b0-4ad9-9cf5-7641377b49c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:06.572533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d01a2c-51b0-4ad9-9cf5-7641377b49c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:06.573330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a39ea2cf-8ee4-4d2d-ba1e-fb4a06f70981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:06.574023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d01a2c-51b0-4ad9-9cf5-7641377b49c7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:06.586572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 27/51] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:06.592618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d01a2c-51b0-4ad9-9cf5-7641377b49c7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:06.643213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 25/52] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:32:06.693743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d01a2c-51b0-4ad9-9cf5-7641377b49c7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:06.703180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:06.727100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080c19a-f088-4eff-98d7-c2aa39892399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:06.728319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080c19a-f088-4eff-98d7-c2aa39892399 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:06.728597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080c19a-f088-4eff-98d7-c2aa39892399 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:06.728871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080c19a-f088-4eff-98d7-c2aa39892399 None admin] PATH_INFO: `/v3/users/ba1856b68bd244e69db7d867dbb38da3` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:06.730157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080c19a-f088-4eff-98d7-c2aa39892399 None admin] RBAC: Authorizing `identity:get_user(user_id=ba1856b68bd244e69db7d867dbb38da3)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:06.731694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080c19a-f088-4eff-98d7-c2aa39892399 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:06.733256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 28/53] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:06 2022] GET /identity/v3/users/ba1856b68bd244e69db7d867dbb38da3 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:06.738813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ea2cf-8ee4-4d2d-ba1e-fb4a06f70981 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:06.833154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:06.839513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ea2cf-8ee4-4d2d-ba1e-fb4a06f70981 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:06.873887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac102f25-fce3-49df-a0ce-3e31f584c261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:06.874521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac102f25-fce3-49df-a0ce-3e31f584c261 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:06.874805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac102f25-fce3-49df-a0ce-3e31f584c261 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:06.875081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac102f25-fce3-49df-a0ce-3e31f584c261 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:06.876193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac102f25-fce3-49df-a0ce-3e31f584c261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:06.877705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac102f25-fce3-49df-a0ce-3e31f584c261 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:06.879293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 26/54] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:06.885811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080c19a-f088-4eff-98d7-c2aa39892399 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:06.910953 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac102f25-fce3-49df-a0ce-3e31f584c261 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:06.986619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080c19a-f088-4eff-98d7-c2aa39892399 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:07.013787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac102f25-fce3-49df-a0ce-3e31f584c261 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:07.021658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8e455-1128-4376-b851-d7f252756b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:07.022306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8e455-1128-4376-b851-d7f252756b2d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:07.022594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8e455-1128-4376-b851-d7f252756b2d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:07.022876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8e455-1128-4376-b851-d7f252756b2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:07.023604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8e455-1128-4376-b851-d7f252756b2d None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.domain.id=default, user.id=ba1856b68bd244e69db7d867dbb38da3) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:07.023917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8e455-1128-4376-b851-d7f252756b2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:07.026007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8e455-1128-4376-b851-d7f252756b2d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:07.034026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 29/55] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Sep 8 18:32:06 2022] GET /identity/v3/role_assignments?user.id=ba1856b68bd244e69db7d867dbb38da3&scope.domain.id=default&role.id=5c777de40b8449329d714d8c2229115e => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:07.048800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa8593-f9e1-4767-ae93-7434110296a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:07.049500 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa8593-f9e1-4767-ae93-7434110296a4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:07.049790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa8593-f9e1-4767-ae93-7434110296a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:07.050071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa8593-f9e1-4767-ae93-7434110296a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:07.051288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa8593-f9e1-4767-ae93-7434110296a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:07.052788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa8593-f9e1-4767-ae93-7434110296a4 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:07.054500 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 27/56] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:07.063639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8e455-1128-4376-b851-d7f252756b2d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:07.109596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:07.171264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8e455-1128-4376-b851-d7f252756b2d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:07.206967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc919904-b116-4236-baa0-d5688d6e6eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:07.207590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc919904-b116-4236-baa0-d5688d6e6eba None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:07.208568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc919904-b116-4236-baa0-d5688d6e6eba None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:07.208568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc919904-b116-4236-baa0-d5688d6e6eba None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:07.208818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc919904-b116-4236-baa0-d5688d6e6eba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:07.210269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc919904-b116-4236-baa0-d5688d6e6eba None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:07.322569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 30/57] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Sep 8 18:32:07 2022] POST /identity/v3/projects => generated 298 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:09.488647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860ee3c-4827-4682-91f0-e56573eb4842 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:09.489200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860ee3c-4827-4682-91f0-e56573eb4842 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:09.489283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860ee3c-4827-4682-91f0-e56573eb4842 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:09.490600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 28/58] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:09.495852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300a571-d71a-423b-b537-25e4de741d0b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:09.496259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300a571-d71a-423b-b537-25e4de741d0b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:09.496397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300a571-d71a-423b-b537-25e4de741d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:09.515178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2300a571-d71a-423b-b537-25e4de741d0b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:09.516362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2300a571-d71a-423b-b537-25e4de741d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:09.580424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 31/59] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:09.847198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860ee3c-4827-4682-91f0-e56573eb4842 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:09.879299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:09.949620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860ee3c-4827-4682-91f0-e56573eb4842 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:09.986126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd7f0d-5a08-4236-8472-c6f3fcd1a5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:09.986759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd7f0d-5a08-4236-8472-c6f3fcd1a5d0 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:09.987039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd7f0d-5a08-4236-8472-c6f3fcd1a5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:09.987313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd7f0d-5a08-4236-8472-c6f3fcd1a5d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:09.988422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd7f0d-5a08-4236-8472-c6f3fcd1a5d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:09.989894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd7f0d-5a08-4236-8472-c6f3fcd1a5d0 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:09.991454 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 29/60] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:09.997775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300a571-d71a-423b-b537-25e4de741d0b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:10.021232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:10.097488 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300a571-d71a-423b-b537-25e4de741d0b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:10.131301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2b267-72c4-49d4-a11a-51e6323f1de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:10.131905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2b267-72c4-49d4-a11a-51e6323f1de7 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:10.132188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2b267-72c4-49d4-a11a-51e6323f1de7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:10.132466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2b267-72c4-49d4-a11a-51e6323f1de7 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:10.133160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2b267-72c4-49d4-a11a-51e6323f1de7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:10.135030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2b267-72c4-49d4-a11a-51e6323f1de7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:10.240671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 32/61] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:32:09 2022] POST /identity/v3/users => generated 275 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:13.438773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005880e-61e4-4346-8267-5b29365184eb None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:13.439813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005880e-61e4-4346-8267-5b29365184eb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:13.440316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005880e-61e4-4346-8267-5b29365184eb None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:13.441905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 30/62] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:13.447815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd368a-cfbd-49f3-8e73-039732c4339a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:13.448629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd368a-cfbd-49f3-8e73-039732c4339a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:13.449149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd368a-cfbd-49f3-8e73-039732c4339a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:13.478588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-26fd368a-cfbd-49f3-8e73-039732c4339a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:13.480114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-26fd368a-cfbd-49f3-8e73-039732c4339a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:13.568645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 33/63] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:32:13.637270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a6d6-6c63-4f10-983a-fe2b03a4cb4e None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:13.637908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a6d6-6c63-4f10-983a-fe2b03a4cb4e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:13.638386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a6d6-6c63-4f10-983a-fe2b03a4cb4e None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:13.640303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 31/64] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:13.645820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e70a4-9522-4e4a-a04a-3acc283dbd28 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:13.646120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e70a4-9522-4e4a-a04a-3acc283dbd28 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:13.646408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e70a4-9522-4e4a-a04a-3acc283dbd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:13.672905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-dd8e70a4-9522-4e4a-a04a-3acc283dbd28 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:13.674153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-dd8e70a4-9522-4e4a-a04a-3acc283dbd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:13.676202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:13.771885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 34/65] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:13.887998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530720a-2571-463a-ad83-6bb7a38a7a50 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:13.888330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530720a-2571-463a-ad83-6bb7a38a7a50 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:13.888567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530720a-2571-463a-ad83-6bb7a38a7a50 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:13.890144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 32/66] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:32:13.895780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d676390-fdca-4038-b240-872cd5501a9a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:13.896288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d676390-fdca-4038-b240-872cd5501a9a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:13.896589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d676390-fdca-4038-b240-872cd5501a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:13.924978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1d676390-fdca-4038-b240-872cd5501a9a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:13.930933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1d676390-fdca-4038-b240-872cd5501a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:14.006402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530720a-2571-463a-ad83-6bb7a38a7a50 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:14.034054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 35/67] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:14.139586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d676390-fdca-4038-b240-872cd5501a9a None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:14.209601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:14.215899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530720a-2571-463a-ad83-6bb7a38a7a50 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:14.286026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:14.291662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b0ee0-d2ee-4aa1-9384-02034976e676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:14.294044 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b0ee0-d2ee-4aa1-9384-02034976e676 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.294535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b0ee0-d2ee-4aa1-9384-02034976e676 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.294957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b0ee0-d2ee-4aa1-9384-02034976e676 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.316811 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b0ee0-d2ee-4aa1-9384-02034976e676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-bc0b0ee0-d2ee-4aa1-9384-02034976e676 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:14.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:14.324163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:14.325094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:14.327091 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 33/68] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:32:14.333647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21318d08-e0eb-48f9-9790-eb7c176fbb1d None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.334216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21318d08-e0eb-48f9-9790-eb7c176fbb1d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.334640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21318d08-e0eb-48f9-9790-eb7c176fbb1d None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.339542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 34/69] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:14.340037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d676390-fdca-4038-b240-872cd5501a9a None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:14.340842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21318d08-e0eb-48f9-9790-eb7c176fbb1d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:14.393583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b6432-fcb4-4f63-9352-c1c903909571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:14.394239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b6432-fcb4-4f63-9352-c1c903909571 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.394524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b6432-fcb4-4f63-9352-c1c903909571 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.394802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b6432-fcb4-4f63-9352-c1c903909571 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.407075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b6432-fcb4-4f63-9352-c1c903909571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-558b6432-fcb4-4f63-9352-c1c903909571 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:14.418085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:14.419023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:14.420502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 36/70] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:32:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:14.421828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877de1c-2b9a-4c13-b759-fbed796520c8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.421828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877de1c-2b9a-4c13-b759-fbed796520c8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.421993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877de1c-2b9a-4c13-b759-fbed796520c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.445039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d877de1c-2b9a-4c13-b759-fbed796520c8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:14.450897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d877de1c-2b9a-4c13-b759-fbed796520c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:14.480673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21318d08-e0eb-48f9-9790-eb7c176fbb1d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:14.526179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339ed48-a228-43e7-a594-3f9c580fa917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:14.526801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339ed48-a228-43e7-a594-3f9c580fa917 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.527080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339ed48-a228-43e7-a594-3f9c580fa917 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.527355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339ed48-a228-43e7-a594-3f9c580fa917 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.528260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339ed48-a228-43e7-a594-3f9c580fa917 None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:14.528566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339ed48-a228-43e7-a594-3f9c580fa917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:14.530002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339ed48-a228-43e7-a594-3f9c580fa917 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:14.536626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 37/71] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:14.537519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877de1c-2b9a-4c13-b759-fbed796520c8 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:14.539165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 35/72] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:14.540416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339ed48-a228-43e7-a594-3f9c580fa917 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:14.583567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:14.626853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:14.684304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877de1c-2b9a-4c13-b759-fbed796520c8 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:14.687885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339ed48-a228-43e7-a594-3f9c580fa917 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:14.731536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef9246-ccbf-41c3-a51e-8466fdf5277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:14.732161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef9246-ccbf-41c3-a51e-8466fdf5277d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.732457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef9246-ccbf-41c3-a51e-8466fdf5277d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.732733 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef9246-ccbf-41c3-a51e-8466fdf5277d None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.733435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef9246-ccbf-41c3-a51e-8466fdf5277d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:14.733543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31294d6a-ec9a-4a23-99a6-44ba4ae695c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:14.733741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef9246-ccbf-41c3-a51e-8466fdf5277d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:14.734089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31294d6a-ec9a-4a23-99a6-44ba4ae695c8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.734373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31294d6a-ec9a-4a23-99a6-44ba4ae695c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.734650 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31294d6a-ec9a-4a23-99a6-44ba4ae695c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.735150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef9246-ccbf-41c3-a51e-8466fdf5277d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:14.745670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 36/73] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:14.747261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d85ce-a550-40ca-a157-75ac2cdf7c9a None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.747550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d85ce-a550-40ca-a157-75ac2cdf7c9a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.747825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d85ce-a550-40ca-a157-75ac2cdf7c9a None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.749268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 37/74] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:14.753168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31294d6a-ec9a-4a23-99a6-44ba4ae695c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:14.754432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d85ce-a550-40ca-a157-75ac2cdf7c9a None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-31294d6a-ec9a-4a23-99a6-44ba4ae695c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:14.758525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:14.759391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:14.760188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 38/75] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:32:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:14.761594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31294d6a-ec9a-4a23-99a6-44ba4ae695c8 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:14.876189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:14.895415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d85ce-a550-40ca-a157-75ac2cdf7c9a None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:14.897486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31294d6a-ec9a-4a23-99a6-44ba4ae695c8 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:14.923006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:14.936021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a3729-0ada-4c5a-b6dc-3560de81efae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:14.936701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a3729-0ada-4c5a-b6dc-3560de81efae None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.936981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a3729-0ada-4c5a-b6dc-3560de81efae None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.937286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a3729-0ada-4c5a-b6dc-3560de81efae None admin] PATH_INFO: `/v3/users/c6ad7503ff7243b2881542fdfef8df5c` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.944574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fedc3-28a0-4a29-8105-03a930dcce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:14.945252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fedc3-28a0-4a29-8105-03a930dcce5c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.945534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fedc3-28a0-4a29-8105-03a930dcce5c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.945809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fedc3-28a0-4a29-8105-03a930dcce5c None admin] PATH_INFO: `/v3/users/ba1856b68bd244e69db7d867dbb38da3` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.948071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12fedc3-28a0-4a29-8105-03a930dcce5c None admin] RBAC: Authorizing `identity:get_user(user_id=ba1856b68bd244e69db7d867dbb38da3)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:14.949609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12fedc3-28a0-4a29-8105-03a930dcce5c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:14.951389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 39/76] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:14 2022] GET /identity/v3/users/ba1856b68bd244e69db7d867dbb38da3 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:14.952501 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc6350-981f-4aaf-8be3-2c931ef38ec9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:14.952885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc6350-981f-4aaf-8be3-2c931ef38ec9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:14.953204 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc6350-981f-4aaf-8be3-2c931ef38ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:14.958937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a3729-0ada-4c5a-b6dc-3560de81efae None admin] RBAC: Authorizing `identity:get_user(user_id=c6ad7503ff7243b2881542fdfef8df5c)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:14.959887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a3729-0ada-4c5a-b6dc-3560de81efae None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:14.964792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 38/77] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:14 2022] GET /identity/v3/users/c6ad7503ff7243b2881542fdfef8df5c => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:14.965660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a3729-0ada-4c5a-b6dc-3560de81efae None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:14.980042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-07dc6350-981f-4aaf-8be3-2c931ef38ec9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:14.981283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-07dc6350-981f-4aaf-8be3-2c931ef38ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:15.058457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 40/78] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:15.059320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc6350-981f-4aaf-8be3-2c931ef38ec9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:15.080316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a3729-0ada-4c5a-b6dc-3560de81efae None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:15.120341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a1b06-8b7b-444d-9e56-f6c965022fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:15.120962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a1b06-8b7b-444d-9e56-f6c965022fda None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:15.121278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a1b06-8b7b-444d-9e56-f6c965022fda None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:15.121546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a1b06-8b7b-444d-9e56-f6c965022fda None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:15.122218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a1b06-8b7b-444d-9e56-f6c965022fda None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:15.122523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a1b06-8b7b-444d-9e56-f6c965022fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:15.123908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a1b06-8b7b-444d-9e56-f6c965022fda None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:15.137868 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 39/79] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:32:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:15.137868 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a1b06-8b7b-444d-9e56-f6c965022fda None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:15.140950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:15.177800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc6350-981f-4aaf-8be3-2c931ef38ec9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:15.195770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:15.215173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7ef61-2a97-4ada-8bc5-05449a220d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:15.215810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7ef61-2a97-4ada-8bc5-05449a220d6a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:15.216088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7ef61-2a97-4ada-8bc5-05449a220d6a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:15.216357 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7ef61-2a97-4ada-8bc5-05449a220d6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:15.226697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7ef61-2a97-4ada-8bc5-05449a220d6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-29e7ef61-2a97-4ada-8bc5-05449a220d6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:15.231879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:15.232747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 41/80] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:15.237061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7ef61-2a97-4ada-8bc5-05449a220d6a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:15.267727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a1b06-8b7b-444d-9e56-f6c965022fda None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:15.311654 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8fbd7-f25a-4471-9b2b-096370c89fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:15.312264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8fbd7-f25a-4471-9b2b-096370c89fdc None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:15.312547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8fbd7-f25a-4471-9b2b-096370c89fdc None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:15.312818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8fbd7-f25a-4471-9b2b-096370c89fdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:15.347108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7ef61-2a97-4ada-8bc5-05449a220d6a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:15.382475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e06074-39a6-44ed-9e5a-69b386dd6eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:15.383091 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e06074-39a6-44ed-9e5a-69b386dd6eee None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:15.383436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e06074-39a6-44ed-9e5a-69b386dd6eee None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:15.384363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e06074-39a6-44ed-9e5a-69b386dd6eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:15.385023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8fbd7-f25a-4471-9b2b-096370c89fdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:15.386562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8fbd7-f25a-4471-9b2b-096370c89fdc None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:15.389262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:15.428718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:15.452215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e06074-39a6-44ed-9e5a-69b386dd6eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:15.453754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e06074-39a6-44ed-9e5a-69b386dd6eee None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:15.486683 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 40/81] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:15.488255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8fbd7-f25a-4471-9b2b-096370c89fdc None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:15.553659 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 42/82] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:15.554514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e06074-39a6-44ed-9e5a-69b386dd6eee None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:15.598799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8fbd7-f25a-4471-9b2b-096370c89fdc None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:15.602229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:15.635227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:15.636268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ad73b-c953-48b2-8da1-fc732071a426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:15.637256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ad73b-c953-48b2-8da1-fc732071a426 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:15.637553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ad73b-c953-48b2-8da1-fc732071a426 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:15.637835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ad73b-c953-48b2-8da1-fc732071a426 None admin] PATH_INFO: `/v3/users/c6ad7503ff7243b2881542fdfef8df5c` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:15.639002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ad73b-c953-48b2-8da1-fc732071a426 None admin] RBAC: Authorizing `identity:get_user(user_id=c6ad7503ff7243b2881542fdfef8df5c)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:15.640510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ad73b-c953-48b2-8da1-fc732071a426 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:15.642094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 41/83] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:15 2022] GET /identity/v3/users/c6ad7503ff7243b2881542fdfef8df5c => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:15.647343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ad73b-c953-48b2-8da1-fc732071a426 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:15.658253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e06074-39a6-44ed-9e5a-69b386dd6eee None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:15.693150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc329a4-ed54-4459-b1af-f9baaf11a433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:15.693763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc329a4-ed54-4459-b1af-f9baaf11a433 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:15.694047 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc329a4-ed54-4459-b1af-f9baaf11a433 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:15.694321 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc329a4-ed54-4459-b1af-f9baaf11a433 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:15.694984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc329a4-ed54-4459-b1af-f9baaf11a433 None admin] RBAC: Adding query filter params (name=member) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:15.695290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc329a4-ed54-4459-b1af-f9baaf11a433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:15.696698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc329a4-ed54-4459-b1af-f9baaf11a433 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:15.704649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 43/84] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:15.705471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc329a4-ed54-4459-b1af-f9baaf11a433 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:15.753957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ad73b-c953-48b2-8da1-fc732071a426 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:15.789778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe7071-3890-4ba0-962c-3d5369bdf90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:15.790407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe7071-3890-4ba0-962c-3d5369bdf90e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:15.790691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe7071-3890-4ba0-962c-3d5369bdf90e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:15.790973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe7071-3890-4ba0-962c-3d5369bdf90e None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:15.792113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe7071-3890-4ba0-962c-3d5369bdf90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:15.793615 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe7071-3890-4ba0-962c-3d5369bdf90e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:15.795207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 42/85] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:15.796002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe7071-3890-4ba0-962c-3d5369bdf90e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:15.806698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc329a4-ed54-4459-b1af-f9baaf11a433 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:15.838332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:15.842605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7f2e3-8692-41da-a096-3f3f0b7dc3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:15.843239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7f2e3-8692-41da-a096-3f3f0b7dc3b5 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:15.843521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7f2e3-8692-41da-a096-3f3f0b7dc3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:15.843799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7f2e3-8692-41da-a096-3f3f0b7dc3b5 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:15.844927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7f2e3-8692-41da-a096-3f3f0b7dc3b5 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:15.846407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7f2e3-8692-41da-a096-3f3f0b7dc3b5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:15.848292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 44/86] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:15 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:15.849080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7f2e3-8692-41da-a096-3f3f0b7dc3b5 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:15.859956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:15.897013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe7071-3890-4ba0-962c-3d5369bdf90e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:15.935348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5056cee-0277-492c-b558-36097e47f7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:15.935990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5056cee-0277-492c-b558-36097e47f7db None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:15.936272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5056cee-0277-492c-b558-36097e47f7db None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:15.936543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5056cee-0277-492c-b558-36097e47f7db None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:15.937693 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5056cee-0277-492c-b558-36097e47f7db None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:15.939140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5056cee-0277-492c-b558-36097e47f7db None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:15.940758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 43/87] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:15 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:15.941555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5056cee-0277-492c-b558-36097e47f7db None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:15.955374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7f2e3-8692-41da-a096-3f3f0b7dc3b5 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:15.989573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9d656-f009-4be6-84b2-ddeeea149355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:15.990184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9d656-f009-4be6-84b2-ddeeea149355 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:15.990469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9d656-f009-4be6-84b2-ddeeea149355 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:15.990751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9d656-f009-4be6-84b2-ddeeea149355 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:16.040582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5056cee-0277-492c-b558-36097e47f7db None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:16.049324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:16.059003 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9d656-f009-4be6-84b2-ddeeea149355 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:16.060493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9d656-f009-4be6-84b2-ddeeea149355 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:16.068163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:16.075804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501ff64-a5b5-480c-b981-c78bf9dacb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:16.076435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501ff64-a5b5-480c-b981-c78bf9dacb4f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:16.076718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501ff64-a5b5-480c-b981-c78bf9dacb4f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:16.076995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501ff64-a5b5-480c-b981-c78bf9dacb4f None admin] PATH_INFO: `/v3/users/c6ad7503ff7243b2881542fdfef8df5c` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:16.078187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501ff64-a5b5-480c-b981-c78bf9dacb4f None admin] RBAC: Authorizing `identity:get_user(user_id=c6ad7503ff7243b2881542fdfef8df5c)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:16.079626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501ff64-a5b5-480c-b981-c78bf9dacb4f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:16.081249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 44/88] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:15 2022] GET /identity/v3/users/c6ad7503ff7243b2881542fdfef8df5c => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:16.081993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501ff64-a5b5-480c-b981-c78bf9dacb4f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:16.160597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 45/89] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:16.161393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9d656-f009-4be6-84b2-ddeeea149355 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:16.186165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501ff64-a5b5-480c-b981-c78bf9dacb4f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:16.219612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a2d42-3283-4c87-8992-be65cfaa6906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:16.220225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a2d42-3283-4c87-8992-be65cfaa6906 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:16.220782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a2d42-3283-4c87-8992-be65cfaa6906 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:16.220782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a2d42-3283-4c87-8992-be65cfaa6906 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:16.221459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a2d42-3283-4c87-8992-be65cfaa6906 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:16.222925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a2d42-3283-4c87-8992-be65cfaa6906 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:16.263173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9d656-f009-4be6-84b2-ddeeea149355 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:16.270214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:16.296665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa5111-71f9-4450-bc6b-adfec74f027b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:16.297299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa5111-71f9-4450-bc6b-adfec74f027b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:16.297581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa5111-71f9-4450-bc6b-adfec74f027b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:16.297856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa5111-71f9-4450-bc6b-adfec74f027b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:16.298573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa5111-71f9-4450-bc6b-adfec74f027b None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=c59969df6296450f916add4011ddf885, user.id=ba1856b68bd244e69db7d867dbb38da3) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:16.298882 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa5111-71f9-4450-bc6b-adfec74f027b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:16.300304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa5111-71f9-4450-bc6b-adfec74f027b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:16.305525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 46/90] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:16 2022] GET /identity/v3/role_assignments?user.id=ba1856b68bd244e69db7d867dbb38da3&scope.project.id=c59969df6296450f916add4011ddf885&role.id=5c777de40b8449329d714d8c2229115e => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:16.306386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa5111-71f9-4450-bc6b-adfec74f027b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:16.340054 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 45/91] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:32:16 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:16.346229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a2d42-3283-4c87-8992-be65cfaa6906 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:16.397456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:16.434793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa5111-71f9-4450-bc6b-adfec74f027b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:16.479756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5a13a-4539-4b3f-9d94-b4c63a269219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:16.480740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5a13a-4539-4b3f-9d94-b4c63a269219 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:16.481135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5a13a-4539-4b3f-9d94-b4c63a269219 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:16.481490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5a13a-4539-4b3f-9d94-b4c63a269219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:16.482382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5a13a-4539-4b3f-9d94-b4c63a269219 None admin] RBAC: Adding query filter params (role.id=eccc3f8721b1466b8a1968b77e5c464f, scope.project.id=c59969df6296450f916add4011ddf885, user.id=c6ad7503ff7243b2881542fdfef8df5c) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:16.482804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5a13a-4539-4b3f-9d94-b4c63a269219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:16.484753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5a13a-4539-4b3f-9d94-b4c63a269219 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:16.491911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 47/92] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:16 2022] GET /identity/v3/role_assignments?user.id=c6ad7503ff7243b2881542fdfef8df5c&scope.project.id=c59969df6296450f916add4011ddf885&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:16.493680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5a13a-4539-4b3f-9d94-b4c63a269219 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:16.505422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a2d42-3283-4c87-8992-be65cfaa6906 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:16.551657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:16.572613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535f71-ab91-4129-9a14-613f6e4093d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:16.573441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535f71-ab91-4129-9a14-613f6e4093d0 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:16.573777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535f71-ab91-4129-9a14-613f6e4093d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:16.574241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535f71-ab91-4129-9a14-613f6e4093d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:16.654473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5a13a-4539-4b3f-9d94-b4c63a269219 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:16.685992 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6535f71-ab91-4129-9a14-613f6e4093d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:16.687753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6535f71-ab91-4129-9a14-613f6e4093d0 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:16.701229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62701df3-5a00-499f-8d05-a32d7ff9aef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:16.702060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62701df3-5a00-499f-8d05-a32d7ff9aef9 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:16.702351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62701df3-5a00-499f-8d05-a32d7ff9aef9 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:16.702633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62701df3-5a00-499f-8d05-a32d7ff9aef9 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:16.703858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62701df3-5a00-499f-8d05-a32d7ff9aef9 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:16.705397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62701df3-5a00-499f-8d05-a32d7ff9aef9 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:16.707399 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 48/93] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:16 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:16.713536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62701df3-5a00-499f-8d05-a32d7ff9aef9 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:16.729946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:16.822425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 46/94] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:16.828153 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535f71-ab91-4129-9a14-613f6e4093d0 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:16.838924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62701df3-5a00-499f-8d05-a32d7ff9aef9 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:16.840740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:16.889642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f51306-d28a-4224-ae6a-67790b108dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:16.889642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f51306-d28a-4224-ae6a-67790b108dfc None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:16.889642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f51306-d28a-4224-ae6a-67790b108dfc None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:16.889642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f51306-d28a-4224-ae6a-67790b108dfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:16.890546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f51306-d28a-4224-ae6a-67790b108dfc None admin] RBAC: Adding query filter params (role.id=6b3a5d54d6044e9b8d8270e7ff3145e8, scope.project.id=c59969df6296450f916add4011ddf885, user.id=c6ad7503ff7243b2881542fdfef8df5c) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:16.890931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f51306-d28a-4224-ae6a-67790b108dfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:16.893429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f51306-d28a-4224-ae6a-67790b108dfc None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:16.901183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 49/95] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:16 2022] GET /identity/v3/role_assignments?user.id=c6ad7503ff7243b2881542fdfef8df5c&scope.project.id=c59969df6296450f916add4011ddf885&role.id=6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:16.991437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535f71-ab91-4129-9a14-613f6e4093d0 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:17.039445 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:17.046451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579c791-532a-48a4-9c74-b48f85d8e889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:17.047105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579c791-532a-48a4-9c74-b48f85d8e889 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:17.047396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579c791-532a-48a4-9c74-b48f85d8e889 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:17.054497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579c791-532a-48a4-9c74-b48f85d8e889 None admin] PATH_INFO: `/v3/projects/34ed033048454636b861cc338026f5ee` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:17.054497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579c791-532a-48a4-9c74-b48f85d8e889 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed033048454636b861cc338026f5ee)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:17.054497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579c791-532a-48a4-9c74-b48f85d8e889 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:17.054497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 47/96] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:16 2022] GET /identity/v3/projects/34ed033048454636b861cc338026f5ee => generated 312 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:17.059272 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f51306-d28a-4224-ae6a-67790b108dfc None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:17.205584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f51306-d28a-4224-ae6a-67790b108dfc None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:17.254770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f299f-50aa-4718-849a-f987bade6d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:17.255319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f299f-50aa-4718-849a-f987bade6d0b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:17.255607 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f299f-50aa-4718-849a-f987bade6d0b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:17.255889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f299f-50aa-4718-849a-f987bade6d0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:17.256623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f299f-50aa-4718-849a-f987bade6d0b None admin] RBAC: Adding query filter params (role.id=eccc3f8721b1466b8a1968b77e5c464f, scope.project.id=34ed033048454636b861cc338026f5ee, user.id=c6ad7503ff7243b2881542fdfef8df5c) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:17.256936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f299f-50aa-4718-849a-f987bade6d0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:17.259007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f299f-50aa-4718-849a-f987bade6d0b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:17.269604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 50/97] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:17 2022] GET /identity/v3/role_assignments?user.id=c6ad7503ff7243b2881542fdfef8df5c&scope.project.id=34ed033048454636b861cc338026f5ee&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:19.175019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89179db7-a846-40b7-a6de-14e511066bd0 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:19.175551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89179db7-a846-40b7-a6de-14e511066bd0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:19.175551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89179db7-a846-40b7-a6de-14e511066bd0 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:19.177088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 48/98] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:19.182677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec53e1-4f0b-479e-b261-d6b027918e69 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:19.183199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec53e1-4f0b-479e-b261-d6b027918e69 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:19.183199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec53e1-4f0b-479e-b261-d6b027918e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:19.209096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c2ec53e1-4f0b-479e-b261-d6b027918e69 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:19.210341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c2ec53e1-4f0b-479e-b261-d6b027918e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:19.241425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:19.302400 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 51/99] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:32:19.534483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ef096-9413-4356-8e2c-55e1a781ceae None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:19.534821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ef096-9413-4356-8e2c-55e1a781ceae None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:19.535060 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ef096-9413-4356-8e2c-55e1a781ceae None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:19.536429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 49/100] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:19.541747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087df42a-1485-4587-bfc1-6ee95aab9185 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:19.542042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087df42a-1485-4587-bfc1-6ee95aab9185 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:19.542326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087df42a-1485-4587-bfc1-6ee95aab9185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:19.565295 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-087df42a-1485-4587-bfc1-6ee95aab9185 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:19.566491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-087df42a-1485-4587-bfc1-6ee95aab9185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:19.669678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 52/101] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:19.838894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ef096-9413-4356-8e2c-55e1a781ceae None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:19.880289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b27ac-1ca4-4ec7-abdc-4f549343d1b4 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:19.880601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b27ac-1ca4-4ec7-abdc-4f549343d1b4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:19.880863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b27ac-1ca4-4ec7-abdc-4f549343d1b4 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:19.882555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 53/102] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:19.889188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c0608-0a93-42d6-a613-021e04c5ec1f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:19.889480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c0608-0a93-42d6-a613-021e04c5ec1f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:19.889761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c0608-0a93-42d6-a613-021e04c5ec1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:19.925377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-580c0608-0a93-42d6-a613-021e04c5ec1f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:19.927010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-580c0608-0a93-42d6-a613-021e04c5ec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:19.992088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ef096-9413-4356-8e2c-55e1a781ceae None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:20.069580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab479ad-f09c-4cce-b05e-14a55a4f5478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:20.070663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab479ad-f09c-4cce-b05e-14a55a4f5478 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.071227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab479ad-f09c-4cce-b05e-14a55a4f5478 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.072071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab479ad-f09c-4cce-b05e-14a55a4f5478 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.084195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab479ad-f09c-4cce-b05e-14a55a4f5478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-7ab479ad-f09c-4cce-b05e-14a55a4f5478 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:20.091350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:20.092544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:20.093349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:20.095640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 50/103] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:32:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:20.097232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab479ad-f09c-4cce-b05e-14a55a4f5478 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:20.166500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 54/104] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:20.167954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c0608-0a93-42d6-a613-021e04c5ec1f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:20.201088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:20.212570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:20.275825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab479ad-f09c-4cce-b05e-14a55a4f5478 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:20.331460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc79f3a-0e74-4d72-bf2f-cd9da0677093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:20.332715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc79f3a-0e74-4d72-bf2f-cd9da0677093 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.333218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc79f3a-0e74-4d72-bf2f-cd9da0677093 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.333676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc79f3a-0e74-4d72-bf2f-cd9da0677093 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.356317 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc79f3a-0e74-4d72-bf2f-cd9da0677093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-4bc79f3a-0e74-4d72-bf2f-cd9da0677093 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:20.363457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:20.364292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:20.365211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:20.369068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 51/105] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:20.372849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d7c4c-3099-46a9-852c-1ba8eea28733 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.373345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d7c4c-3099-46a9-852c-1ba8eea28733 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.375335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d7c4c-3099-46a9-852c-1ba8eea28733 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.377428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 52/106] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:20.378321 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d7c4c-3099-46a9-852c-1ba8eea28733 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:20.382328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c0608-0a93-42d6-a613-021e04c5ec1f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:20.456494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f28011-1df0-4cfd-b4eb-260fd856ae69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:20.458089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28011-1df0-4cfd-b4eb-260fd856ae69 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.460063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28011-1df0-4cfd-b4eb-260fd856ae69 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.460517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28011-1df0-4cfd-b4eb-260fd856ae69 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.461509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f28011-1df0-4cfd-b4eb-260fd856ae69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:20.462005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f28011-1df0-4cfd-b4eb-260fd856ae69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:20.463759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f28011-1df0-4cfd-b4eb-260fd856ae69 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:20.474198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 55/107] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:20.475902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44469a-fa6e-468a-969f-93f946588ca1 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.476336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44469a-fa6e-468a-969f-93f946588ca1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.476772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44469a-fa6e-468a-969f-93f946588ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.511365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7c44469a-fa6e-468a-969f-93f946588ca1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:20.513636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7c44469a-fa6e-468a-969f-93f946588ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:20.569325 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d7c4c-3099-46a9-852c-1ba8eea28733 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:20.596928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:20.597334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 56/108] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:20.598207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44469a-fa6e-468a-969f-93f946588ca1 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:20.615071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367ff84-d573-44c2-a458-83f53ef556fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:20.615690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367ff84-d573-44c2-a458-83f53ef556fa None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.615969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367ff84-d573-44c2-a458-83f53ef556fa None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.616245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367ff84-d573-44c2-a458-83f53ef556fa None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.616912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367ff84-d573-44c2-a458-83f53ef556fa None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:20.617118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:20.617273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367ff84-d573-44c2-a458-83f53ef556fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:20.619280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367ff84-d573-44c2-a458-83f53ef556fa None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:20.628655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 53/109] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:20.630326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367ff84-d573-44c2-a458-83f53ef556fa None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:20.716821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44469a-fa6e-468a-969f-93f946588ca1 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:20.735312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367ff84-d573-44c2-a458-83f53ef556fa None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:20.753522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cc628-2278-4273-8a2c-ff65307e7c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:20.754157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cc628-2278-4273-8a2c-ff65307e7c82 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.754441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cc628-2278-4273-8a2c-ff65307e7c82 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.754718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cc628-2278-4273-8a2c-ff65307e7c82 None admin] PATH_INFO: `/v3/users/ba1856b68bd244e69db7d867dbb38da3` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.755840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731cc628-2278-4273-8a2c-ff65307e7c82 None admin] RBAC: Authorizing `identity:get_user(user_id=ba1856b68bd244e69db7d867dbb38da3)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:20.757304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731cc628-2278-4273-8a2c-ff65307e7c82 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:20.763745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 57/110] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:20 2022] GET /identity/v3/users/ba1856b68bd244e69db7d867dbb38da3 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:20.763745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a989-eb12-4d2c-9b26-ad102cb86824 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.763745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a989-eb12-4d2c-9b26-ad102cb86824 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.763745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92a989-eb12-4d2c-9b26-ad102cb86824 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.763745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 58/111] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:20.767728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed81a05-7f6f-4899-95c0-108e29f3dbb3 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.768022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed81a05-7f6f-4899-95c0-108e29f3dbb3 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.768306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed81a05-7f6f-4899-95c0-108e29f3dbb3 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.769568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 59/112] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:20.770343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed81a05-7f6f-4899-95c0-108e29f3dbb3 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:20.780780 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd614416-8d6a-4257-83c0-03cb31f9fcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:20.781521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd614416-8d6a-4257-83c0-03cb31f9fcad None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.781867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd614416-8d6a-4257-83c0-03cb31f9fcad None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.782159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd614416-8d6a-4257-83c0-03cb31f9fcad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.793893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd614416-8d6a-4257-83c0-03cb31f9fcad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-dd614416-8d6a-4257-83c0-03cb31f9fcad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:20.799432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:20.800279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:20.801180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 54/113] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:32:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:20.801829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd614416-8d6a-4257-83c0-03cb31f9fcad None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:20.847936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:20.876521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:20.896709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed81a05-7f6f-4899-95c0-108e29f3dbb3 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:20.906708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd614416-8d6a-4257-83c0-03cb31f9fcad None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:20.932081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2b22d-299a-459a-9ff4-682bdde055f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:20.932710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2b22d-299a-459a-9ff4-682bdde055f4 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.932990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2b22d-299a-459a-9ff4-682bdde055f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.933290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2b22d-299a-459a-9ff4-682bdde055f4 None admin] PATH_INFO: `/v3/users/c6ad7503ff7243b2881542fdfef8df5c` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.934418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2b22d-299a-459a-9ff4-682bdde055f4 None admin] RBAC: Authorizing `identity:get_user(user_id=c6ad7503ff7243b2881542fdfef8df5c)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:20.935848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2b22d-299a-459a-9ff4-682bdde055f4 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:20.937466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 60/114] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:20 2022] GET /identity/v3/users/c6ad7503ff7243b2881542fdfef8df5c => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:20.938558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee50a4-ea09-4805-8156-3d2b59d727a6 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.938841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee50a4-ea09-4805-8156-3d2b59d727a6 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.939114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee50a4-ea09-4805-8156-3d2b59d727a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.941142 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadea17e-c2f7-44a1-9f0d-e0cfa2cb9803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:20.941780 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadea17e-c2f7-44a1-9f0d-e0cfa2cb9803 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.942066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadea17e-c2f7-44a1-9f0d-e0cfa2cb9803 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.942343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadea17e-c2f7-44a1-9f0d-e0cfa2cb9803 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.943452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadea17e-c2f7-44a1-9f0d-e0cfa2cb9803 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:20.945002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadea17e-c2f7-44a1-9f0d-e0cfa2cb9803 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:20.947337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 55/115] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:20 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:20.953026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1643-2f9e-415a-a817-2bcb8676f7a6 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:20.954412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1643-2f9e-415a-a817-2bcb8676f7a6 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:20.954412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a1643-2f9e-415a-a817-2bcb8676f7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:20.957725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d6ee50a4-ea09-4805-8156-3d2b59d727a6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:20.958913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d6ee50a4-ea09-4805-8156-3d2b59d727a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:20.972314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-093a1643-2f9e-415a-a817-2bcb8676f7a6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:20.973523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-093a1643-2f9e-415a-a817-2bcb8676f7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:21.026854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 61/116] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:21.027691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee50a4-ea09-4805-8156-3d2b59d727a6 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.040350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 56/117] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:21.047614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a1643-2f9e-415a-a817-2bcb8676f7a6 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.100956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:21.155752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee50a4-ea09-4805-8156-3d2b59d727a6 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:21.157471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:21.176593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a1643-2f9e-415a-a817-2bcb8676f7a6 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:21.195722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9d2ae-c62d-4db6-9eba-e4ab0f7a9865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:21.197640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9d2ae-c62d-4db6-9eba-e4ab0f7a9865 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:21.198170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9d2ae-c62d-4db6-9eba-e4ab0f7a9865 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:21.198637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9d2ae-c62d-4db6-9eba-e4ab0f7a9865 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:21.199492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9d2ae-c62d-4db6-9eba-e4ab0f7a9865 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:21.199989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9d2ae-c62d-4db6-9eba-e4ab0f7a9865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:21.201702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9d2ae-c62d-4db6-9eba-e4ab0f7a9865 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:21.210171 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 62/118] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:32:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:21.211192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9d2ae-c62d-4db6-9eba-e4ab0f7a9865 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.220760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4b8f7-8f09-43d9-b296-a117c854548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:21.221437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4b8f7-8f09-43d9-b296-a117c854548b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:21.221727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4b8f7-8f09-43d9-b296-a117c854548b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:21.222007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4b8f7-8f09-43d9-b296-a117c854548b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:21.233845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4b8f7-8f09-43d9-b296-a117c854548b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-6df4b8f7-8f09-43d9-b296-a117c854548b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:21.239429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:21.240277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:21.241035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 57/119] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:21.241704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4b8f7-8f09-43d9-b296-a117c854548b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.320429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9d2ae-c62d-4db6-9eba-e4ab0f7a9865 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:21.354840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f5c97-b208-4db6-a8f8-fed1d81e7f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:21.355503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5c97-b208-4db6-a8f8-fed1d81e7f82 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:21.355783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5c97-b208-4db6-a8f8-fed1d81e7f82 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:21.356060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5c97-b208-4db6-a8f8-fed1d81e7f82 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:21.357280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f5c97-b208-4db6-a8f8-fed1d81e7f82 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:21.357383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:21.358642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f5c97-b208-4db6-a8f8-fed1d81e7f82 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:21.360132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4b8f7-8f09-43d9-b296-a117c854548b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:21.360263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 63/120] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:21 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:21.361031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f5c97-b208-4db6-a8f8-fed1d81e7f82 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.372529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:21.397103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb782f59-2a3a-4df1-8709-97087f722347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:21.397853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb782f59-2a3a-4df1-8709-97087f722347 None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:21.398140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb782f59-2a3a-4df1-8709-97087f722347 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:21.398417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb782f59-2a3a-4df1-8709-97087f722347 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885/users/ba1856b68bd244e69db7d867dbb38da3/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:21.400581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb782f59-2a3a-4df1-8709-97087f722347 None admin] RBAC: Authorizing `identity:create_grant(project_id=c59969df6296450f916add4011ddf885, user_id=ba1856b68bd244e69db7d867dbb38da3, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:21.402092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb782f59-2a3a-4df1-8709-97087f722347 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:21.460970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f5c97-b208-4db6-a8f8-fed1d81e7f82 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:21.474391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 58/121] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:32:21 2022] PUT /identity/v3/projects/c59969df6296450f916add4011ddf885/users/ba1856b68bd244e69db7d867dbb38da3/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:32:21.475294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb782f59-2a3a-4df1-8709-97087f722347 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.498594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880db51-7745-4e49-af41-d1688e4b4fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:21.499512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880db51-7745-4e49-af41-d1688e4b4fd0 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:21.499944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880db51-7745-4e49-af41-d1688e4b4fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:21.500396 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880db51-7745-4e49-af41-d1688e4b4fd0 None admin] PATH_INFO: `/v3/users/c6ad7503ff7243b2881542fdfef8df5c` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:21.501890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0880db51-7745-4e49-af41-d1688e4b4fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=c6ad7503ff7243b2881542fdfef8df5c)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:21.503578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0880db51-7745-4e49-af41-d1688e4b4fd0 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:21.506508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 64/122] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:21 2022] GET /identity/v3/users/c6ad7503ff7243b2881542fdfef8df5c => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:21.507551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880db51-7745-4e49-af41-d1688e4b4fd0 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.599634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb782f59-2a3a-4df1-8709-97087f722347 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:21.606556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:21.633394 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880db51-7745-4e49-af41-d1688e4b4fd0 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:21.641930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78e14c-6837-42c2-820b-3936d7ed134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:21.642649 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78e14c-6837-42c2-820b-3936d7ed134a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:21.643010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78e14c-6837-42c2-820b-3936d7ed134a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:21.643375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78e14c-6837-42c2-820b-3936d7ed134a None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:21.644137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78e14c-6837-42c2-820b-3936d7ed134a None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:21.644452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78e14c-6837-42c2-820b-3936d7ed134a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:21.645962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78e14c-6837-42c2-820b-3936d7ed134a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:21.655783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 59/123] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:21.656174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78e14c-6837-42c2-820b-3936d7ed134a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.681609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0224b-e26f-462e-9c7e-95684487a200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:21.682258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0224b-e26f-462e-9c7e-95684487a200 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:21.682552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0224b-e26f-462e-9c7e-95684487a200 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:21.682834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0224b-e26f-462e-9c7e-95684487a200 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:21.683974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0224b-e26f-462e-9c7e-95684487a200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:21.685478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0224b-e26f-462e-9c7e-95684487a200 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:21.686230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:21.687157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 65/124] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:21.687983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0224b-e26f-462e-9c7e-95684487a200 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.772673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78e14c-6837-42c2-820b-3936d7ed134a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:21.795839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0224b-e26f-462e-9c7e-95684487a200 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:21.815714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e8552-0cc4-4929-884c-ec8b08223e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:21.816355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e8552-0cc4-4929-884c-ec8b08223e1f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:21.816641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e8552-0cc4-4929-884c-ec8b08223e1f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:21.816977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e8552-0cc4-4929-884c-ec8b08223e1f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:21.831342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e8552-0cc4-4929-884c-ec8b08223e1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:21.831485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6aadb3-fde9-4b6d-bd2d-7b3eed38dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:21.832044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aadb3-fde9-4b6d-bd2d-7b3eed38dc61 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:21.832328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aadb3-fde9-4b6d-bd2d-7b3eed38dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:21.832606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6aadb3-fde9-4b6d-bd2d-7b3eed38dc61 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885/users/c6ad7503ff7243b2881542fdfef8df5c/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:21.834532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6aadb3-fde9-4b6d-bd2d-7b3eed38dc61 None admin] RBAC: Authorizing `identity:create_grant(project_id=c59969df6296450f916add4011ddf885, user_id=c6ad7503ff7243b2881542fdfef8df5c, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:21.835972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6aadb3-fde9-4b6d-bd2d-7b3eed38dc61 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-372e8552-0cc4-4929-884c-ec8b08223e1f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:21.836274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:21.837132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:21.837983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 60/125] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:21.839453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e8552-0cc4-4929-884c-ec8b08223e1f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.922084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 66/126] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:32:21 2022] PUT /identity/v3/projects/c59969df6296450f916add4011ddf885/users/c6ad7503ff7243b2881542fdfef8df5c/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:32:21.923309 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6aadb3-fde9-4b6d-bd2d-7b3eed38dc61 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:21.928362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:21.933195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:21.966010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e8552-0cc4-4929-884c-ec8b08223e1f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:22.022463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c55a9-d76a-44a9-8bf4-6e051eef3b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:22.023564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c55a9-d76a-44a9-8bf4-6e051eef3b4c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:22.024688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c55a9-d76a-44a9-8bf4-6e051eef3b4c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:22.025921 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c55a9-d76a-44a9-8bf4-6e051eef3b4c None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:22.027537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c55a9-d76a-44a9-8bf4-6e051eef3b4c None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:22.031144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c55a9-d76a-44a9-8bf4-6e051eef3b4c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:22.038932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 61/127] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:21 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:22.038932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c55a9-d76a-44a9-8bf4-6e051eef3b4c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:22.067407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6aadb3-fde9-4b6d-bd2d-7b3eed38dc61 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:22.114351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7796e2-6f04-4eb4-8a3d-83a66efd2809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:22.115006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7796e2-6f04-4eb4-8a3d-83a66efd2809 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:22.115344 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7796e2-6f04-4eb4-8a3d-83a66efd2809 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:22.115629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7796e2-6f04-4eb4-8a3d-83a66efd2809 None admin] PATH_INFO: `/v3/users/c6ad7503ff7243b2881542fdfef8df5c` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:22.116791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7796e2-6f04-4eb4-8a3d-83a66efd2809 None admin] RBAC: Authorizing `identity:get_user(user_id=c6ad7503ff7243b2881542fdfef8df5c)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:22.118454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7796e2-6f04-4eb4-8a3d-83a66efd2809 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:22.120749 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 67/128] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:21 2022] GET /identity/v3/users/c6ad7503ff7243b2881542fdfef8df5c => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:22.121903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7796e2-6f04-4eb4-8a3d-83a66efd2809 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:22.188269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c55a9-d76a-44a9-8bf4-6e051eef3b4c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:22.215382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:22.237296 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faba22-b06a-4956-9125-71eae5384180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:22.238064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faba22-b06a-4956-9125-71eae5384180 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:22.238357 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faba22-b06a-4956-9125-71eae5384180 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:22.238966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faba22-b06a-4956-9125-71eae5384180 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:22.240090 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faba22-b06a-4956-9125-71eae5384180 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:22.243203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faba22-b06a-4956-9125-71eae5384180 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:22.249909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7796e2-6f04-4eb4-8a3d-83a66efd2809 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:22.286651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffac66-7847-4f82-af4a-fa8f7a157167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:22.287280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffac66-7847-4f82-af4a-fa8f7a157167 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:22.287566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffac66-7847-4f82-af4a-fa8f7a157167 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:22.287847 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffac66-7847-4f82-af4a-fa8f7a157167 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:22.288518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffac66-7847-4f82-af4a-fa8f7a157167 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:22.288824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffac66-7847-4f82-af4a-fa8f7a157167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:22.290459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffac66-7847-4f82-af4a-fa8f7a157167 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:22.299474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 68/129] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:22.300281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffac66-7847-4f82-af4a-fa8f7a157167 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:22.331018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 62/130] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:32:22 2022] POST /identity/v3/projects => generated 302 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:22.333019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faba22-b06a-4956-9125-71eae5384180 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:22.338347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:22.468409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faba22-b06a-4956-9125-71eae5384180 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:22.470531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffac66-7847-4f82-af4a-fa8f7a157167 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:22.507924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8520b-4066-43e5-8c65-03c88ad6f6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:22.509730 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8520b-4066-43e5-8c65-03c88ad6f6c2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:22.510396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8520b-4066-43e5-8c65-03c88ad6f6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:22.510872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8520b-4066-43e5-8c65-03c88ad6f6c2 None admin] PATH_INFO: `/v3/projects/34ed033048454636b861cc338026f5ee` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:22.512256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8520b-4066-43e5-8c65-03c88ad6f6c2 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed033048454636b861cc338026f5ee)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:22.514015 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8520b-4066-43e5-8c65-03c88ad6f6c2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:22.515969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 63/131] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:22 2022] GET /identity/v3/projects/34ed033048454636b861cc338026f5ee => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:22.517202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8520b-4066-43e5-8c65-03c88ad6f6c2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:22.528984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bab15b-69f8-4695-ac7c-36798f3702a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:22.530037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bab15b-69f8-4695-ac7c-36798f3702a6 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:22.530548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bab15b-69f8-4695-ac7c-36798f3702a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:22.531027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bab15b-69f8-4695-ac7c-36798f3702a6 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885/users/c6ad7503ff7243b2881542fdfef8df5c/roles/6b3a5d54d6044e9b8d8270e7ff3145e8` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:22.533640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bab15b-69f8-4695-ac7c-36798f3702a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c59969df6296450f916add4011ddf885, user_id=c6ad7503ff7243b2881542fdfef8df5c, role_id=6b3a5d54d6044e9b8d8270e7ff3145e8)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:22.536019 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bab15b-69f8-4695-ac7c-36798f3702a6 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:22.539358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:22.614792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 69/132] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:32:22 2022] PUT /identity/v3/projects/c59969df6296450f916add4011ddf885/users/c6ad7503ff7243b2881542fdfef8df5c/roles/6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:32:22.615886 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bab15b-69f8-4695-ac7c-36798f3702a6 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:22.624596 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:22.696755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8520b-4066-43e5-8c65-03c88ad6f6c2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:22.757811 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b0f48-fcb4-4322-93dd-c14893a1ca5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:22.759230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0f48-fcb4-4322-93dd-c14893a1ca5f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:22.759729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0f48-fcb4-4322-93dd-c14893a1ca5f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:22.760210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0f48-fcb4-4322-93dd-c14893a1ca5f None admin] PATH_INFO: `/v3/users/586479a9c9ed4428b2f6c9bde72351ba` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:22.781752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b0f48-fcb4-4322-93dd-c14893a1ca5f None admin] RBAC: Authorizing `identity:get_user(user_id=586479a9c9ed4428b2f6c9bde72351ba)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:22.783593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b0f48-fcb4-4322-93dd-c14893a1ca5f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:22.785886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 64/133] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:22 2022] GET /identity/v3/users/586479a9c9ed4428b2f6c9bde72351ba => generated 289 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:22.793331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b0f48-fcb4-4322-93dd-c14893a1ca5f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:22.809515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bab15b-69f8-4695-ac7c-36798f3702a6 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:22.862537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d7018-b5bd-4c57-aa51-e0b16a2aa63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:22.863292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d7018-b5bd-4c57-aa51-e0b16a2aa63f None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:22.863582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d7018-b5bd-4c57-aa51-e0b16a2aa63f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:22.863864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d7018-b5bd-4c57-aa51-e0b16a2aa63f None admin] PATH_INFO: `/v3/projects/34ed033048454636b861cc338026f5ee/users/c6ad7503ff7243b2881542fdfef8df5c/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:22.866022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d7018-b5bd-4c57-aa51-e0b16a2aa63f None admin] RBAC: Authorizing `identity:create_grant(project_id=34ed033048454636b861cc338026f5ee, user_id=c6ad7503ff7243b2881542fdfef8df5c, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:22.867558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d7018-b5bd-4c57-aa51-e0b16a2aa63f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:22.912724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b0f48-fcb4-4322-93dd-c14893a1ca5f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:22.944489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:22.953064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bf3af-08d4-41d3-ab3d-62469e497f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:22.953781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bf3af-08d4-41d3-ab3d-62469e497f8a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:22.954074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bf3af-08d4-41d3-ab3d-62469e497f8a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:22.954364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bf3af-08d4-41d3-ab3d-62469e497f8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:22.994787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 70/134] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:32:22 2022] PUT /identity/v3/projects/34ed033048454636b861cc338026f5ee/users/c6ad7503ff7243b2881542fdfef8df5c/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:32:23.046236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bf3af-08d4-41d3-ab3d-62469e497f8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:23.047735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bf3af-08d4-41d3-ab3d-62469e497f8a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:23.163994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 65/135] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:23.169997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d7018-b5bd-4c57-aa51-e0b16a2aa63f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:23.255145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:23.310863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d7018-b5bd-4c57-aa51-e0b16a2aa63f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:23.357598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f2428-efff-4669-ad5f-8823dcc40e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:23.358259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f2428-efff-4669-ad5f-8823dcc40e6f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:23.358554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f2428-efff-4669-ad5f-8823dcc40e6f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:23.358833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f2428-efff-4669-ad5f-8823dcc40e6f None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:23.360029 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f2428-efff-4669-ad5f-8823dcc40e6f None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:23.361522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f2428-efff-4669-ad5f-8823dcc40e6f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:23.363183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 71/136] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:23 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:23.372937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bf3af-08d4-41d3-ab3d-62469e497f8a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:23.410739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:23.503453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bf3af-08d4-41d3-ab3d-62469e497f8a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:23.555765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00029f-8fe9-4f4e-9b71-34ef8c4d7773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:23.556819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00029f-8fe9-4f4e-9b71-34ef8c4d7773 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:23.557275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00029f-8fe9-4f4e-9b71-34ef8c4d7773 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:23.557724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00029f-8fe9-4f4e-9b71-34ef8c4d7773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:23.558605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00029f-8fe9-4f4e-9b71-34ef8c4d7773 None admin] RBAC: Adding query filter params (role.id=1714afd0de144939a3022b0632f03aa0, scope.project.id=c59969df6296450f916add4011ddf885, user.id=586479a9c9ed4428b2f6c9bde72351ba) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:23.559065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00029f-8fe9-4f4e-9b71-34ef8c4d7773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:23.560669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00029f-8fe9-4f4e-9b71-34ef8c4d7773 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:23.567376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 66/137] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:23 2022] GET /identity/v3/role_assignments?user.id=586479a9c9ed4428b2f6c9bde72351ba&scope.project.id=c59969df6296450f916add4011ddf885&role.id=1714afd0de144939a3022b0632f03aa0 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:24.023612 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38295283-8a63-451d-b9cb-58658161e470 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:24.024818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38295283-8a63-451d-b9cb-58658161e470 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:24.025329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38295283-8a63-451d-b9cb-58658161e470 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:24.026885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 72/138] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:24.046527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e7d6c-add6-4492-b806-0d9c4658142d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:24.046834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e7d6c-add6-4492-b806-0d9c4658142d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:24.047100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e7d6c-add6-4492-b806-0d9c4658142d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:24.074114 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-023e7d6c-add6-4492-b806-0d9c4658142d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:24.075363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-023e7d6c-add6-4492-b806-0d9c4658142d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:24.167720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 67/139] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:24.489915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38295283-8a63-451d-b9cb-58658161e470 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:24.625903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38295283-8a63-451d-b9cb-58658161e470 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:24.655694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:24.667841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fb3e9-66e3-449d-8d0f-f1e55191c95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:24.668495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fb3e9-66e3-449d-8d0f-f1e55191c95a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:24.668778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fb3e9-66e3-449d-8d0f-f1e55191c95a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:24.669086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fb3e9-66e3-449d-8d0f-f1e55191c95a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:24.680686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fb3e9-66e3-449d-8d0f-f1e55191c95a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-585fb3e9-66e3-449d-8d0f-f1e55191c95a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:24.685676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:24.686519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:24.687359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 73/140] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:32:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:24.692797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e7d6c-add6-4492-b806-0d9c4658142d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:24.719261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:24.815759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e7d6c-add6-4492-b806-0d9c4658142d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:24.865793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd416edb-4611-4a0d-941f-de4841097b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:24.866747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd416edb-4611-4a0d-941f-de4841097b57 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:24.867228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd416edb-4611-4a0d-941f-de4841097b57 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:24.867738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd416edb-4611-4a0d-941f-de4841097b57 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:24.868696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd416edb-4611-4a0d-941f-de4841097b57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:24.873282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd416edb-4611-4a0d-941f-de4841097b57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:24.874941 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd416edb-4611-4a0d-941f-de4841097b57 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:24.885083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 68/141] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:24.890784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fb3e9-66e3-449d-8d0f-f1e55191c95a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:25.034194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fb3e9-66e3-449d-8d0f-f1e55191c95a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:25.081934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1479c-eea0-4f12-865d-5eefdd2dd24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:25.082497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1479c-eea0-4f12-865d-5eefdd2dd24e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:25.082777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1479c-eea0-4f12-865d-5eefdd2dd24e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:25.083052 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1479c-eea0-4f12-865d-5eefdd2dd24e None admin] PATH_INFO: `/v3/users/ba1856b68bd244e69db7d867dbb38da3` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:25.089979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1479c-eea0-4f12-865d-5eefdd2dd24e None admin] RBAC: Authorizing `identity:get_user(user_id=ba1856b68bd244e69db7d867dbb38da3)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:25.092719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1479c-eea0-4f12-865d-5eefdd2dd24e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:25.095722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 74/142] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:24 2022] GET /identity/v3/users/ba1856b68bd244e69db7d867dbb38da3 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:25.100831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd416edb-4611-4a0d-941f-de4841097b57 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:25.136371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2578e-773e-4fc9-8bf6-916ec3307f1e None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:25.136686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2578e-773e-4fc9-8bf6-916ec3307f1e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:25.136923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2578e-773e-4fc9-8bf6-916ec3307f1e None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:25.138543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 75/143] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:25.172205 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17913b-081a-4924-844a-074a66cbac16 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:25.172500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17913b-081a-4924-844a-074a66cbac16 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:25.172770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17913b-081a-4924-844a-074a66cbac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:25.201952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-ef17913b-081a-4924-844a-074a66cbac16 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:25.203157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ef17913b-081a-4924-844a-074a66cbac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:25.251756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:25.279402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd416edb-4611-4a0d-941f-de4841097b57 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:25.310743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:25.347329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fb015-0ff1-4f95-ab2c-161c43af3d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:25.348433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fb015-0ff1-4f95-ab2c-161c43af3d7f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:25.350088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fb015-0ff1-4f95-ab2c-161c43af3d7f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:25.350890 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fb015-0ff1-4f95-ab2c-161c43af3d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:25.355868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 76/144] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:32:25.440184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fb015-0ff1-4f95-ab2c-161c43af3d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:25.442049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fb015-0ff1-4f95-ab2c-161c43af3d7f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:25.542815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:25.566791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 69/145] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:25.573343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17913b-081a-4924-844a-074a66cbac16 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:25.691151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17913b-081a-4924-844a-074a66cbac16 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:25.721633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fb015-0ff1-4f95-ab2c-161c43af3d7f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:25.726851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793add1f-5b0c-4d75-9735-908e3d10663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:25.728118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793add1f-5b0c-4d75-9735-908e3d10663d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:25.728683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793add1f-5b0c-4d75-9735-908e3d10663d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:25.728958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793add1f-5b0c-4d75-9735-908e3d10663d None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:25.730172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793add1f-5b0c-4d75-9735-908e3d10663d None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:25.731626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793add1f-5b0c-4d75-9735-908e3d10663d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:25.733613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 77/146] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:25 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:25.739301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793add1f-5b0c-4d75-9735-908e3d10663d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:25.797494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:25.858169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793add1f-5b0c-4d75-9735-908e3d10663d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:25.895947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf69706-f8f7-4fa4-9331-3f0f226130da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:25.896563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf69706-f8f7-4fa4-9331-3f0f226130da None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:25.896846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf69706-f8f7-4fa4-9331-3f0f226130da None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:25.897151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf69706-f8f7-4fa4-9331-3f0f226130da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:25.897992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf69706-f8f7-4fa4-9331-3f0f226130da None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=c59969df6296450f916add4011ddf885, user.id=ba1856b68bd244e69db7d867dbb38da3) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:25.898246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf69706-f8f7-4fa4-9331-3f0f226130da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:25.900395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf69706-f8f7-4fa4-9331-3f0f226130da None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:25.907801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 78/147] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:25 2022] GET /identity/v3/role_assignments?user.id=ba1856b68bd244e69db7d867dbb38da3&scope.project.id=c59969df6296450f916add4011ddf885&role.id=5c777de40b8449329d714d8c2229115e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:25.932916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fb015-0ff1-4f95-ab2c-161c43af3d7f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:25.986978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3597c9-ecbb-4d79-ae8d-8891ffa247f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:25.988027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3597c9-ecbb-4d79-ae8d-8891ffa247f4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:25.988504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3597c9-ecbb-4d79-ae8d-8891ffa247f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:25.988972 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3597c9-ecbb-4d79-ae8d-8891ffa247f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.014244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3597c9-ecbb-4d79-ae8d-8891ffa247f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-dc3597c9-ecbb-4d79-ae8d-8891ffa247f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:26.019698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:26.020578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:26.021374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:26.024354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 70/148] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:26.030500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf69706-f8f7-4fa4-9331-3f0f226130da None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:26.072376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fc841-0aa4-4f1f-bfe8-ac4ddda6c061 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:26.073022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fc841-0aa4-4f1f-bfe8-ac4ddda6c061 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:26.073537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fc841-0aa4-4f1f-bfe8-ac4ddda6c061 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.075230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 71/149] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:32:26.081171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f097f-aaab-4f51-98fd-ff0da1935659 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:26.081768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f097f-aaab-4f51-98fd-ff0da1935659 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:26.082237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f097f-aaab-4f51-98fd-ff0da1935659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.122242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c07f097f-aaab-4f51-98fd-ff0da1935659 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:26.123485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c07f097f-aaab-4f51-98fd-ff0da1935659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:26.194358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:26.229134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 72/150] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:32:26.235578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf69706-f8f7-4fa4-9331-3f0f226130da None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:26.253871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:26.287155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d79bb9-f0cb-462f-8e1a-9a5276ef8bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:26.288215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d79bb9-f0cb-462f-8e1a-9a5276ef8bd7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:26.288760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d79bb9-f0cb-462f-8e1a-9a5276ef8bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:26.289264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d79bb9-f0cb-462f-8e1a-9a5276ef8bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.290710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d79bb9-f0cb-462f-8e1a-9a5276ef8bd7 None admin] RBAC: Adding query filter params (name=member) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:26.291269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d79bb9-f0cb-462f-8e1a-9a5276ef8bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:26.292978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d79bb9-f0cb-462f-8e1a-9a5276ef8bd7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:26.303626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 79/151] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:26.310011 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f097f-aaab-4f51-98fd-ff0da1935659 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:26.412596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203a6cd-9d1d-4825-8c5a-fdd2b5711e09 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:26.412790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203a6cd-9d1d-4825-8c5a-fdd2b5711e09 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:26.413116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203a6cd-9d1d-4825-8c5a-fdd2b5711e09 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.415665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 80/152] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:26.421476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb277-5d81-4448-a19d-466c029c467f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:26.421777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb277-5d81-4448-a19d-466c029c467f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:26.422021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb277-5d81-4448-a19d-466c029c467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.443239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-5a7fb277-5d81-4448-a19d-466c029c467f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:26.444773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5a7fb277-5d81-4448-a19d-466c029c467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:26.455199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f097f-aaab-4f51-98fd-ff0da1935659 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:26.530378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558427ae-0479-4ef1-a004-2c16db66ff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:26.531691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558427ae-0479-4ef1-a004-2c16db66ff9b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:26.532178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558427ae-0479-4ef1-a004-2c16db66ff9b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:26.532650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558427ae-0479-4ef1-a004-2c16db66ff9b None admin] PATH_INFO: `/v3/users/c6ad7503ff7243b2881542fdfef8df5c` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.535780 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558427ae-0479-4ef1-a004-2c16db66ff9b None admin] RBAC: Authorizing `identity:get_user(user_id=c6ad7503ff7243b2881542fdfef8df5c)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:26.537579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558427ae-0479-4ef1-a004-2c16db66ff9b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:26.539981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 73/153] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:26 2022] GET /identity/v3/users/c6ad7503ff7243b2881542fdfef8df5c => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:26.541621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1022fc-36ca-4209-a092-a2c489b54be4 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:26.542132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1022fc-36ca-4209-a092-a2c489b54be4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:26.542854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1022fc-36ca-4209-a092-a2c489b54be4 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.544313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 74/154] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:26.550411 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b86c38-27a5-48e1-a39f-7e389837707a None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:26.551699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b86c38-27a5-48e1-a39f-7e389837707a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:26.551989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b86c38-27a5-48e1-a39f-7e389837707a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.570170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 81/155] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:26.571435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fb277-5d81-4448-a19d-466c029c467f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:26.582293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-63b86c38-27a5-48e1-a39f-7e389837707a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:26.587957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-63b86c38-27a5-48e1-a39f-7e389837707a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:26.643548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:26.696381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 75/156] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:26.705067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b86c38-27a5-48e1-a39f-7e389837707a None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:26.722248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fb277-5d81-4448-a19d-466c029c467f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:26.741405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:26.788852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b4735-6715-4ce5-9352-48a2db94383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:26.789638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b4735-6715-4ce5-9352-48a2db94383c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:26.790070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b4735-6715-4ce5-9352-48a2db94383c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:26.790664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b4735-6715-4ce5-9352-48a2db94383c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.865913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b4735-6715-4ce5-9352-48a2db94383c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:26.867451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b4735-6715-4ce5-9352-48a2db94383c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:26.879126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b86c38-27a5-48e1-a39f-7e389837707a None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:26.944358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac21ed-7941-4898-8c74-4089d50d19f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:26.945074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac21ed-7941-4898-8c74-4089d50d19f9 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:26.945371 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac21ed-7941-4898-8c74-4089d50d19f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:26.946305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac21ed-7941-4898-8c74-4089d50d19f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:26.947636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac21ed-7941-4898-8c74-4089d50d19f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:26.949160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac21ed-7941-4898-8c74-4089d50d19f9 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:26.951016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 76/157] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:26.951739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac21ed-7941-4898-8c74-4089d50d19f9 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:26.972112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 82/158] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:26.972846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b4735-6715-4ce5-9352-48a2db94383c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:26.976793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:27.006917 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:27.074125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac21ed-7941-4898-8c74-4089d50d19f9 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:27.084513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b4735-6715-4ce5-9352-48a2db94383c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:27.110572 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb009bf-9e40-41bd-9795-1813738cca93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:27.111203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb009bf-9e40-41bd-9795-1813738cca93 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.111481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb009bf-9e40-41bd-9795-1813738cca93 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.111756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb009bf-9e40-41bd-9795-1813738cca93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:27.122311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb009bf-9e40-41bd-9795-1813738cca93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:27.123074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac7cfa-163c-46b6-8349-925252569e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:27.123709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac7cfa-163c-46b6-8349-925252569e43 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.123989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac7cfa-163c-46b6-8349-925252569e43 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.124258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac7cfa-163c-46b6-8349-925252569e43 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:27.124904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac7cfa-163c-46b6-8349-925252569e43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:27.126372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac7cfa-163c-46b6-8349-925252569e43 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-2bb009bf-9e40-41bd-9795-1813738cca93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:27.127530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:27.128347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 77/159] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:32:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:27.134938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb009bf-9e40-41bd-9795-1813738cca93 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:27.232585 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:27.235402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb009bf-9e40-41bd-9795-1813738cca93 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:27.256707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 83/160] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:32:26 2022] POST /identity/v3/users => generated 283 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:27.257497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac7cfa-163c-46b6-8349-925252569e43 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:27.269676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b858b72-3b52-43ee-9c27-d1fc80c45871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:27.270341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b858b72-3b52-43ee-9c27-d1fc80c45871 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.270624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b858b72-3b52-43ee-9c27-d1fc80c45871 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.270920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b858b72-3b52-43ee-9c27-d1fc80c45871 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:27.272126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b858b72-3b52-43ee-9c27-d1fc80c45871 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:27.273661 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b858b72-3b52-43ee-9c27-d1fc80c45871 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:27.276989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 78/161] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:27 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:27.277773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b858b72-3b52-43ee-9c27-d1fc80c45871 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:27.349491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:27.397579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b858b72-3b52-43ee-9c27-d1fc80c45871 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:27.397963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac7cfa-163c-46b6-8349-925252569e43 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:27.434905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c02bba-cb8a-40c6-a275-c8ab81f76356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:27.435683 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c02bba-cb8a-40c6-a275-c8ab81f76356 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.436137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c02bba-cb8a-40c6-a275-c8ab81f76356 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.436599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c02bba-cb8a-40c6-a275-c8ab81f76356 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:27.437825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c02bba-cb8a-40c6-a275-c8ab81f76356 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:27.438338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c02bba-cb8a-40c6-a275-c8ab81f76356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:27.440512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c02bba-cb8a-40c6-a275-c8ab81f76356 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:27.441571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997acc1b-ea89-4200-a3ae-ba421445cdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:27.442220 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997acc1b-ea89-4200-a3ae-ba421445cdb7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.442508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997acc1b-ea89-4200-a3ae-ba421445cdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.442804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997acc1b-ea89-4200-a3ae-ba421445cdb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:27.458345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 79/162] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:32:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:27.458345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ceed4-f12b-4fae-8de3-d40ebbe2ad05 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.458345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ceed4-f12b-4fae-8de3-d40ebbe2ad05 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.458345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ceed4-f12b-4fae-8de3-d40ebbe2ad05 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:27.459220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 80/163] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:27.460487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997acc1b-ea89-4200-a3ae-ba421445cdb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:27.460607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ceed4-f12b-4fae-8de3-d40ebbe2ad05 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-997acc1b-ea89-4200-a3ae-ba421445cdb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:27.466284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:27.467456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:27.468305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 84/164] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:27.469100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997acc1b-ea89-4200-a3ae-ba421445cdb7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:27.503308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:27.598494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997acc1b-ea89-4200-a3ae-ba421445cdb7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:27.599069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ceed4-f12b-4fae-8de3-d40ebbe2ad05 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:27.623870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:27.640395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51260f8d-1b9e-4e82-bf65-b76533a9ae9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:27.641125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51260f8d-1b9e-4e82-bf65-b76533a9ae9a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.641403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51260f8d-1b9e-4e82-bf65-b76533a9ae9a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.641678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51260f8d-1b9e-4e82-bf65-b76533a9ae9a None admin] PATH_INFO: `/v3/users/c6ad7503ff7243b2881542fdfef8df5c` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:27.643186 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51260f8d-1b9e-4e82-bf65-b76533a9ae9a None admin] RBAC: Authorizing `identity:get_user(user_id=c6ad7503ff7243b2881542fdfef8df5c)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:27.644725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51260f8d-1b9e-4e82-bf65-b76533a9ae9a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:27.645869 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b91b3-46fb-4909-b26e-71580ef04899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:27.646227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b91b3-46fb-4909-b26e-71580ef04899 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.646600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 85/165] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:27 2022] GET /identity/v3/users/c6ad7503ff7243b2881542fdfef8df5c => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:27.646701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b91b3-46fb-4909-b26e-71580ef04899 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.646817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b91b3-46fb-4909-b26e-71580ef04899 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:27.647500 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b91b3-46fb-4909-b26e-71580ef04899 None admin] RBAC: Adding query filter params (role.id=eccc3f8721b1466b8a1968b77e5c464f, scope.project.id=c59969df6296450f916add4011ddf885, user.id=c6ad7503ff7243b2881542fdfef8df5c) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:27.647811 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b91b3-46fb-4909-b26e-71580ef04899 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:27.649941 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b91b3-46fb-4909-b26e-71580ef04899 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:27.650754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa863e-fd1e-4952-97c9-e7cc0b9ec9d9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.651227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa863e-fd1e-4952-97c9-e7cc0b9ec9d9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.651702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa863e-fd1e-4952-97c9-e7cc0b9ec9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:27.657995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 81/166] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:27 2022] GET /identity/v3/role_assignments?user.id=c6ad7503ff7243b2881542fdfef8df5c&scope.project.id=c59969df6296450f916add4011ddf885&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:27.666304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b91b3-46fb-4909-b26e-71580ef04899 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:27.675811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4eaa863e-fd1e-4952-97c9-e7cc0b9ec9d9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:27.677567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4eaa863e-fd1e-4952-97c9-e7cc0b9ec9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:27.778558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 86/167] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:27.780563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa863e-fd1e-4952-97c9-e7cc0b9ec9d9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:27.792711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:27.801359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b91b3-46fb-4909-b26e-71580ef04899 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:27.856447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26b90d-23a7-4fce-af8c-5a8620b6d600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:27.857212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26b90d-23a7-4fce-af8c-5a8620b6d600 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.857493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26b90d-23a7-4fce-af8c-5a8620b6d600 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.857770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26b90d-23a7-4fce-af8c-5a8620b6d600 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:27.858471 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26b90d-23a7-4fce-af8c-5a8620b6d600 None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:27.858753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26b90d-23a7-4fce-af8c-5a8620b6d600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:27.860345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26b90d-23a7-4fce-af8c-5a8620b6d600 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:27.873416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 82/168] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:27.879426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26b90d-23a7-4fce-af8c-5a8620b6d600 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:27.935721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa863e-fd1e-4952-97c9-e7cc0b9ec9d9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:27.961198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:27.988953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356170b-2b74-4742-9376-2ae89a296278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:27.990002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356170b-2b74-4742-9376-2ae89a296278 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:27.990479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356170b-2b74-4742-9376-2ae89a296278 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:27.990946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356170b-2b74-4742-9376-2ae89a296278 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:28.034050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26b90d-23a7-4fce-af8c-5a8620b6d600 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:28.083040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566ddfb-7e7d-42aa-9d9b-d48e1e941777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:28.083719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566ddfb-7e7d-42aa-9d9b-d48e1e941777 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:28.084001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566ddfb-7e7d-42aa-9d9b-d48e1e941777 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:28.084275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566ddfb-7e7d-42aa-9d9b-d48e1e941777 None admin] PATH_INFO: `/v3/users/c6ad7503ff7243b2881542fdfef8df5c` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:28.086200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566ddfb-7e7d-42aa-9d9b-d48e1e941777 None admin] RBAC: Authorizing `identity:get_user(user_id=c6ad7503ff7243b2881542fdfef8df5c)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:28.088844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566ddfb-7e7d-42aa-9d9b-d48e1e941777 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:28.091385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 83/169] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:27 2022] GET /identity/v3/users/c6ad7503ff7243b2881542fdfef8df5c => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:28.095309 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356170b-2b74-4742-9376-2ae89a296278 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:28.096589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566ddfb-7e7d-42aa-9d9b-d48e1e941777 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:28.097108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356170b-2b74-4742-9376-2ae89a296278 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:28.141636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:28.232186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566ddfb-7e7d-42aa-9d9b-d48e1e941777 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:28.244618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 87/170] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:28.250034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356170b-2b74-4742-9376-2ae89a296278 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:28.273644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:28.282272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4d7bd-a8e8-44f4-92cd-2ba75971447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:28.282917 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4d7bd-a8e8-44f4-92cd-2ba75971447d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:28.283171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4d7bd-a8e8-44f4-92cd-2ba75971447d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:28.283438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4d7bd-a8e8-44f4-92cd-2ba75971447d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:28.402976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4d7bd-a8e8-44f4-92cd-2ba75971447d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:28.405352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4d7bd-a8e8-44f4-92cd-2ba75971447d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:28.407604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356170b-2b74-4742-9376-2ae89a296278 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:28.465922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:28.473190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeb6eb-8ef5-4175-8af7-3e62b3d8a1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:28.474912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeb6eb-8ef5-4175-8af7-3e62b3d8a1b3 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:28.476806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeb6eb-8ef5-4175-8af7-3e62b3d8a1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:28.487655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeb6eb-8ef5-4175-8af7-3e62b3d8a1b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:28.507301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efeb6eb-8ef5-4175-8af7-3e62b3d8a1b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-7efeb6eb-8ef5-4175-8af7-3e62b3d8a1b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:28.516189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:28.517116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:28.517912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:28.532016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 88/171] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:28.533322 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeb6eb-8ef5-4175-8af7-3e62b3d8a1b3 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:28.593166 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 84/172] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:28.594472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4d7bd-a8e8-44f4-92cd-2ba75971447d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:28.710858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:28.732588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4d7bd-a8e8-44f4-92cd-2ba75971447d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:28.746536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeb6eb-8ef5-4175-8af7-3e62b3d8a1b3 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:28.781422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:28.799899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb054a75-6826-485e-b3fb-c8158e8324f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:28.800514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb054a75-6826-485e-b3fb-c8158e8324f0 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:28.800797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb054a75-6826-485e-b3fb-c8158e8324f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:28.801513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb054a75-6826-485e-b3fb-c8158e8324f0 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:28.802212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb054a75-6826-485e-b3fb-c8158e8324f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:28.802526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb054a75-6826-485e-b3fb-c8158e8324f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:28.806026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb054a75-6826-485e-b3fb-c8158e8324f0 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:28.815733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c37db5-b1fb-4d3d-b1c5-660c4bf62f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:28.816375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c37db5-b1fb-4d3d-b1c5-660c4bf62f6c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:28.816653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c37db5-b1fb-4d3d-b1c5-660c4bf62f6c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:28.816926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c37db5-b1fb-4d3d-b1c5-660c4bf62f6c None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:28.818966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c37db5-b1fb-4d3d-b1c5-660c4bf62f6c None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:28.820480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c37db5-b1fb-4d3d-b1c5-660c4bf62f6c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:28.824494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 85/174] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:28.825376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb054a75-6826-485e-b3fb-c8158e8324f0 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:28.830167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 89/174] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:28 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:28.831555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c37db5-b1fb-4d3d-b1c5-660c4bf62f6c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:28.974024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb054a75-6826-485e-b3fb-c8158e8324f0 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:29.035559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c37db5-b1fb-4d3d-b1c5-660c4bf62f6c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:29.039157 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56abf7-c838-4251-9223-2812bbbc8f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:29.040102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56abf7-c838-4251-9223-2812bbbc8f69 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:29.040603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56abf7-c838-4251-9223-2812bbbc8f69 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:29.041102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56abf7-c838-4251-9223-2812bbbc8f69 None admin] PATH_INFO: `/v3/projects/34ed033048454636b861cc338026f5ee` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:29.043136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56abf7-c838-4251-9223-2812bbbc8f69 None admin] RBAC: Authorizing `identity:get_project(project_id=34ed033048454636b861cc338026f5ee)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:29.044802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56abf7-c838-4251-9223-2812bbbc8f69 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:29.046670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 86/175] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:28 2022] GET /identity/v3/projects/34ed033048454636b861cc338026f5ee => generated 312 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:29.052242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56abf7-c838-4251-9223-2812bbbc8f69 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:29.107390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c889c4-ef79-4920-b2fd-b87920f94112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:29.108388 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c889c4-ef79-4920-b2fd-b87920f94112 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:29.108861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c889c4-ef79-4920-b2fd-b87920f94112 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:29.109362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c889c4-ef79-4920-b2fd-b87920f94112 None admin] PATH_INFO: `/v3/users/586479a9c9ed4428b2f6c9bde72351ba` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:29.110813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c889c4-ef79-4920-b2fd-b87920f94112 None admin] RBAC: Authorizing `identity:get_user(user_id=586479a9c9ed4428b2f6c9bde72351ba)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:29.114356 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c889c4-ef79-4920-b2fd-b87920f94112 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:29.116877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 90/176] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:28 2022] GET /identity/v3/users/586479a9c9ed4428b2f6c9bde72351ba => generated 289 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:29.119230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c889c4-ef79-4920-b2fd-b87920f94112 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:29.150897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:29.153413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:29.230490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56abf7-c838-4251-9223-2812bbbc8f69 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:29.298900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171568a-5ae0-4897-bcf0-b28a7a79a402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:29.299522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171568a-5ae0-4897-bcf0-b28a7a79a402 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:29.299802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171568a-5ae0-4897-bcf0-b28a7a79a402 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:29.300073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171568a-5ae0-4897-bcf0-b28a7a79a402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:29.300804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171568a-5ae0-4897-bcf0-b28a7a79a402 None admin] RBAC: Adding query filter params (role.id=6b3a5d54d6044e9b8d8270e7ff3145e8, scope.project.id=c59969df6296450f916add4011ddf885, user.id=c6ad7503ff7243b2881542fdfef8df5c) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:29.301133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171568a-5ae0-4897-bcf0-b28a7a79a402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:29.304220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171568a-5ae0-4897-bcf0-b28a7a79a402 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:29.317212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 87/177] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:29 2022] GET /identity/v3/role_assignments?user.id=c6ad7503ff7243b2881542fdfef8df5c&scope.project.id=c59969df6296450f916add4011ddf885&role.id=6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:29.318360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171568a-5ae0-4897-bcf0-b28a7a79a402 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:29.322599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c889c4-ef79-4920-b2fd-b87920f94112 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:29.403800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80dfdd-b985-4f35-a56e-8c4b5a9f4d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:29.404802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80dfdd-b985-4f35-a56e-8c4b5a9f4d9a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:29.407314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80dfdd-b985-4f35-a56e-8c4b5a9f4d9a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:29.407871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80dfdd-b985-4f35-a56e-8c4b5a9f4d9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:29.409176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80dfdd-b985-4f35-a56e-8c4b5a9f4d9a None admin] RBAC: Adding query filter params (role.id=eccc3f8721b1466b8a1968b77e5c464f, scope.project.id=34ed033048454636b861cc338026f5ee, user.id=c6ad7503ff7243b2881542fdfef8df5c) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:29.411015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80dfdd-b985-4f35-a56e-8c4b5a9f4d9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:29.412689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80dfdd-b985-4f35-a56e-8c4b5a9f4d9a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:29.426715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 91/178] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:29 2022] GET /identity/v3/role_assignments?user.id=c6ad7503ff7243b2881542fdfef8df5c&scope.project.id=34ed033048454636b861cc338026f5ee&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:29.481373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171568a-5ae0-4897-bcf0-b28a7a79a402 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:29.510790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:29.543052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71e713-a6eb-4a22-976b-d9a5e314a18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:29.544258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71e713-a6eb-4a22-976b-d9a5e314a18a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:29.544740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71e713-a6eb-4a22-976b-d9a5e314a18a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:29.545238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71e713-a6eb-4a22-976b-d9a5e314a18a None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:29.546701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71e713-a6eb-4a22-976b-d9a5e314a18a None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:29.548429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71e713-a6eb-4a22-976b-d9a5e314a18a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:29.550485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 88/179] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:29 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:29.557502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80dfdd-b985-4f35-a56e-8c4b5a9f4d9a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:29.653556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:29.680693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80dfdd-b985-4f35-a56e-8c4b5a9f4d9a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:29.720338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4373f3-5247-4140-a75a-b640b20bbc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:29.721017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4373f3-5247-4140-a75a-b640b20bbc76 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:29.721349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4373f3-5247-4140-a75a-b640b20bbc76 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:29.721629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4373f3-5247-4140-a75a-b640b20bbc76 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885/users/586479a9c9ed4428b2f6c9bde72351ba/roles/1714afd0de144939a3022b0632f03aa0` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:29.723702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4373f3-5247-4140-a75a-b640b20bbc76 None admin] RBAC: Authorizing `identity:create_grant(project_id=c59969df6296450f916add4011ddf885, user_id=586479a9c9ed4428b2f6c9bde72351ba, role_id=1714afd0de144939a3022b0632f03aa0)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:29.725282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4373f3-5247-4140-a75a-b640b20bbc76 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:29.800574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 92/180] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:32:29 2022] PUT /identity/v3/projects/c59969df6296450f916add4011ddf885/users/586479a9c9ed4428b2f6c9bde72351ba/roles/1714afd0de144939a3022b0632f03aa0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:32:29.928893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d826c40-6007-4d00-b4f9-893f81535e94 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:29.929586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d826c40-6007-4d00-b4f9-893f81535e94 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:29.930104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d826c40-6007-4d00-b4f9-893f81535e94 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:29.931795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 89/181] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:29.938009 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf226f-bc15-4c98-b203-361639624e96 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:29.938288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf226f-bc15-4c98-b203-361639624e96 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:29.938569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf226f-bc15-4c98-b203-361639624e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:29.966702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-edcf226f-bc15-4c98-b203-361639624e96 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:29.968195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-edcf226f-bc15-4c98-b203-361639624e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:30.057595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 93/182] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:30.325774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d826c40-6007-4d00-b4f9-893f81535e94 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:30.443237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d826c40-6007-4d00-b4f9-893f81535e94 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:30.480977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e754c7-ac99-4b59-92c9-bf4dd1b915ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:30.481727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e754c7-ac99-4b59-92c9-bf4dd1b915ce None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:30.482016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e754c7-ac99-4b59-92c9-bf4dd1b915ce None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:30.482293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e754c7-ac99-4b59-92c9-bf4dd1b915ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:30.496379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e754c7-ac99-4b59-92c9-bf4dd1b915ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-31e754c7-ac99-4b59-92c9-bf4dd1b915ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:30.501595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:30.502533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:30.503574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 90/183] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:32:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:30.514295 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf226f-bc15-4c98-b203-361639624e96 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:30.584999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:30.607001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa75b2-4fd8-4e09-a094-18c9893b8c0e None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:30.607287 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa75b2-4fd8-4e09-a094-18c9893b8c0e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:30.607559 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa75b2-4fd8-4e09-a094-18c9893b8c0e None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:30.609264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 91/184] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:30.614525 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdca9b-91aa-4b5f-b1a9-158f9c276127 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:30.614813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdca9b-91aa-4b5f-b1a9-158f9c276127 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:30.615094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdca9b-91aa-4b5f-b1a9-158f9c276127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:30.626317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf226f-bc15-4c98-b203-361639624e96 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:30.637555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-24cdca9b-91aa-4b5f-b1a9-158f9c276127 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:30.642271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-24cdca9b-91aa-4b5f-b1a9-158f9c276127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:30.664439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04864424-abc4-4cce-833d-6e4e288f7bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:30.665413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04864424-abc4-4cce-833d-6e4e288f7bb9 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:30.665785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04864424-abc4-4cce-833d-6e4e288f7bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:30.666153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04864424-abc4-4cce-833d-6e4e288f7bb9 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:30.666975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04864424-abc4-4cce-833d-6e4e288f7bb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:30.667374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04864424-abc4-4cce-833d-6e4e288f7bb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:30.668830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04864424-abc4-4cce-833d-6e4e288f7bb9 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:30.682696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 94/185] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:30.684353 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04864424-abc4-4cce-833d-6e4e288f7bb9 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:30.699472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:30.739923 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 92/186] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:30.799401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04864424-abc4-4cce-833d-6e4e288f7bb9 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:30.823177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67189edb-e19d-4f51-93dc-d956f0beddcc None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:30.823802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67189edb-e19d-4f51-93dc-d956f0beddcc None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:30.824308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67189edb-e19d-4f51-93dc-d956f0beddcc None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:30.826268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 93/187] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:30.832484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f6719-b2cc-415e-8dc4-f0fe4db93d67 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:30.833001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:30.833774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f6719-b2cc-415e-8dc4-f0fe4db93d67 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:30.834345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f6719-b2cc-415e-8dc4-f0fe4db93d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:30.840246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad22b0-0bb9-4d11-ab1b-2b6bc9dbee47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:30.840820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad22b0-0bb9-4d11-ab1b-2b6bc9dbee47 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:30.841119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad22b0-0bb9-4d11-ab1b-2b6bc9dbee47 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:30.841393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad22b0-0bb9-4d11-ab1b-2b6bc9dbee47 None admin] PATH_INFO: `/v3/users/f660125dad194f68bde4d7d3dad65811` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:30.858260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad22b0-0bb9-4d11-ab1b-2b6bc9dbee47 None admin] RBAC: Authorizing `identity:get_user(user_id=f660125dad194f68bde4d7d3dad65811)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:30.860222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad22b0-0bb9-4d11-ab1b-2b6bc9dbee47 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:30.860509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b50f6719-b2cc-415e-8dc4-f0fe4db93d67 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:30.861759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b50f6719-b2cc-415e-8dc4-f0fe4db93d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:30.862640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 95/188] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:30 2022] GET /identity/v3/users/f660125dad194f68bde4d7d3dad65811 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:30.870522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad22b0-0bb9-4d11-ab1b-2b6bc9dbee47 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:30.961504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 94/189] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:30.989883 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad22b0-0bb9-4d11-ab1b-2b6bc9dbee47 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:31.024916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f6719-b2cc-415e-8dc4-f0fe4db93d67 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:31.029226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c7735-cc4b-42c0-bcc8-55a05bd424e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:31.029853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c7735-cc4b-42c0-bcc8-55a05bd424e0 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:31.030134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c7735-cc4b-42c0-bcc8-55a05bd424e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:31.030466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c7735-cc4b-42c0-bcc8-55a05bd424e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:31.093828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:31.108926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c7735-cc4b-42c0-bcc8-55a05bd424e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:31.110722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c7735-cc4b-42c0-bcc8-55a05bd424e0 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:31.143991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f6719-b2cc-415e-8dc4-f0fe4db93d67 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:31.152779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:31.187770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944871d1-34d2-41ce-ac3b-2e4249691d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:31.188401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944871d1-34d2-41ce-ac3b-2e4249691d06 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:31.188681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944871d1-34d2-41ce-ac3b-2e4249691d06 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:31.188951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944871d1-34d2-41ce-ac3b-2e4249691d06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:31.199620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944871d1-34d2-41ce-ac3b-2e4249691d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-944871d1-34d2-41ce-ac3b-2e4249691d06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:31.204573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:31.205746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:31.206817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 95/190] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:32:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:32:31.211862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944871d1-34d2-41ce-ac3b-2e4249691d06 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:31.237959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 96/191] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:31.244196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c7735-cc4b-42c0-bcc8-55a05bd424e0 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:31.321004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944871d1-34d2-41ce-ac3b-2e4249691d06 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:31.344498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:31.357194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c7735-cc4b-42c0-bcc8-55a05bd424e0 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:31.358879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a0458-d588-4b17-bd21-8ceea20b3c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:31.359505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a0458-d588-4b17-bd21-8ceea20b3c29 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:31.359796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a0458-d588-4b17-bd21-8ceea20b3c29 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:31.360153 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a0458-d588-4b17-bd21-8ceea20b3c29 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:31.360834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a0458-d588-4b17-bd21-8ceea20b3c29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:31.361173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a0458-d588-4b17-bd21-8ceea20b3c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:31.362558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a0458-d588-4b17-bd21-8ceea20b3c29 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:31.376950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 96/192] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:32:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:31.376950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a0458-d588-4b17-bd21-8ceea20b3c29 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:31.395048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bd70b-15b4-41e0-b00c-edaf6cf533cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:31.395683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bd70b-15b4-41e0-b00c-edaf6cf533cb None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:31.395962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bd70b-15b4-41e0-b00c-edaf6cf533cb None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:31.396261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bd70b-15b4-41e0-b00c-edaf6cf533cb None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:31.397371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bd70b-15b4-41e0-b00c-edaf6cf533cb None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:31.399376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bd70b-15b4-41e0-b00c-edaf6cf533cb None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:31.401172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 97/193] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:31 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:31.401778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bd70b-15b4-41e0-b00c-edaf6cf533cb None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:31.411776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:31.486197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a0458-d588-4b17-bd21-8ceea20b3c29 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:31.507450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bd70b-15b4-41e0-b00c-edaf6cf533cb None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:31.520110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f31979-e92e-4a8a-82b6-67dead794e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:31.520741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f31979-e92e-4a8a-82b6-67dead794e59 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:31.521024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f31979-e92e-4a8a-82b6-67dead794e59 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:31.521323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f31979-e92e-4a8a-82b6-67dead794e59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:31.522432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f31979-e92e-4a8a-82b6-67dead794e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:31.523965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f31979-e92e-4a8a-82b6-67dead794e59 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:31.529212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 97/194] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:31.529212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f31979-e92e-4a8a-82b6-67dead794e59 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:31.547364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957cdc1-449c-4d3b-a291-948106d297f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:31.548028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957cdc1-449c-4d3b-a291-948106d297f8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:31.548286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957cdc1-449c-4d3b-a291-948106d297f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:31.548565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957cdc1-449c-4d3b-a291-948106d297f8 None admin] PATH_INFO: `/v3/users/f660125dad194f68bde4d7d3dad65811` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:31.549726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957cdc1-449c-4d3b-a291-948106d297f8 None admin] RBAC: Authorizing `identity:get_user(user_id=f660125dad194f68bde4d7d3dad65811)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:31.551162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957cdc1-449c-4d3b-a291-948106d297f8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:31.552755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 98/195] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:31 2022] GET /identity/v3/users/f660125dad194f68bde4d7d3dad65811 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:31.553535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957cdc1-449c-4d3b-a291-948106d297f8 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:31.580662 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:31.644899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:31.651294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f31979-e92e-4a8a-82b6-67dead794e59 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:31.667167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957cdc1-449c-4d3b-a291-948106d297f8 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:31.688145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e2a3e-0273-4c04-92cf-9f1ae371bc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:31.689151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e2a3e-0273-4c04-92cf-9f1ae371bc4f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:31.689436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e2a3e-0273-4c04-92cf-9f1ae371bc4f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:31.689707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e2a3e-0273-4c04-92cf-9f1ae371bc4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:31.690444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e2a3e-0273-4c04-92cf-9f1ae371bc4f None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377, user.id=f660125dad194f68bde4d7d3dad65811) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:31.690756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e2a3e-0273-4c04-92cf-9f1ae371bc4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:31.692201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e2a3e-0273-4c04-92cf-9f1ae371bc4f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:31.707850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 98/196] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:31 2022] GET /identity/v3/role_assignments?user.id=f660125dad194f68bde4d7d3dad65811&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=5c777de40b8449329d714d8c2229115e => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:31.707850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e2a3e-0273-4c04-92cf-9f1ae371bc4f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:31.714989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356cc7-9d75-4ff0-ab54-9237fe8f68b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:31.715686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356cc7-9d75-4ff0-ab54-9237fe8f68b5 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:31.716449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356cc7-9d75-4ff0-ab54-9237fe8f68b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:31.716449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356cc7-9d75-4ff0-ab54-9237fe8f68b5 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:31.717079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96356cc7-9d75-4ff0-ab54-9237fe8f68b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:31.719315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96356cc7-9d75-4ff0-ab54-9237fe8f68b5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:31.814331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 99/197] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:32:31 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:31.835842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e2a3e-0273-4c04-92cf-9f1ae371bc4f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:31.884989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb8850-e2c4-4aae-a24a-9b500141b32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:31.885621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8850-e2c4-4aae-a24a-9b500141b32d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:31.885910 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8850-e2c4-4aae-a24a-9b500141b32d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:31.886185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb8850-e2c4-4aae-a24a-9b500141b32d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:31.923901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:31.975272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb8850-e2c4-4aae-a24a-9b500141b32d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:31.976812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb8850-e2c4-4aae-a24a-9b500141b32d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:32.099049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 99/198] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:32.106292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356cc7-9d75-4ff0-ab54-9237fe8f68b5 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:32.172324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218f96f-afe7-49db-ac3e-abea697ac573 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:32.172578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218f96f-afe7-49db-ac3e-abea697ac573 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:32.172992 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218f96f-afe7-49db-ac3e-abea697ac573 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:32.174451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 100/199] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:32:32.179817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e371-1c85-4ac2-a3d2-89386a156bde None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:32.180521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e371-1c85-4ac2-a3d2-89386a156bde None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:32.182145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e371-1c85-4ac2-a3d2-89386a156bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:32.207550 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:32.209214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f280e371-1c85-4ac2-a3d2-89386a156bde None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:32.210429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f280e371-1c85-4ac2-a3d2-89386a156bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:32.237579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356cc7-9d75-4ff0-ab54-9237fe8f68b5 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:32.285742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cac95e-e746-4ee1-8186-a6387a516198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:32.285976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:32.286449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cac95e-e746-4ee1-8186-a6387a516198 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:32.286741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cac95e-e746-4ee1-8186-a6387a516198 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:32.287016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cac95e-e746-4ee1-8186-a6387a516198 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:32.288392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cac95e-e746-4ee1-8186-a6387a516198 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:32.290777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cac95e-e746-4ee1-8186-a6387a516198 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:32.292443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 100/200] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:32 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:32.298574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cac95e-e746-4ee1-8186-a6387a516198 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:32.302549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 101/201] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:32.412871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cac95e-e746-4ee1-8186-a6387a516198 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:32.449895 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19dc28-1bc3-464d-8c79-4c7fc4c7acf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:32.450833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19dc28-1bc3-464d-8c79-4c7fc4c7acf3 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:32.451315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19dc28-1bc3-464d-8c79-4c7fc4c7acf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:32.451781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19dc28-1bc3-464d-8c79-4c7fc4c7acf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:32.452803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19dc28-1bc3-464d-8c79-4c7fc4c7acf3 None admin] RBAC: Adding query filter params (role.id=6b3a5d54d6044e9b8d8270e7ff3145e8, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377, user.id=f660125dad194f68bde4d7d3dad65811) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:32.453352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19dc28-1bc3-464d-8c79-4c7fc4c7acf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:32.455011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19dc28-1bc3-464d-8c79-4c7fc4c7acf3 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:32.461954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 101/202] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:32 2022] GET /identity/v3/role_assignments?user.id=f660125dad194f68bde4d7d3dad65811&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:32.597908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e371-1c85-4ac2-a3d2-89386a156bde None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:32.715710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e371-1c85-4ac2-a3d2-89386a156bde None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:32.763614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c0d14-3ac0-4a43-a3a0-8413d05a722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:32.764547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c0d14-3ac0-4a43-a3a0-8413d05a722d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:32.765026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c0d14-3ac0-4a43-a3a0-8413d05a722d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:32.765523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c0d14-3ac0-4a43-a3a0-8413d05a722d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:32.782745 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c0d14-3ac0-4a43-a3a0-8413d05a722d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-d66c0d14-3ac0-4a43-a3a0-8413d05a722d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:32.788258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:32.789388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:32.790307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:32.792315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 102/203] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:32.798249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19dc28-1bc3-464d-8c79-4c7fc4c7acf3 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:32.815768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:32.918805 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19dc28-1bc3-464d-8c79-4c7fc4c7acf3 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:32.965942 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eca882-f848-4fcc-8f7f-9d1fd90011af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:32.966537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca882-f848-4fcc-8f7f-9d1fd90011af None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:32.966824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca882-f848-4fcc-8f7f-9d1fd90011af None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:32.967098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca882-f848-4fcc-8f7f-9d1fd90011af None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:32.968032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eca882-f848-4fcc-8f7f-9d1fd90011af None admin] RBAC: Adding query filter params (name=reader) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:32.968337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eca882-f848-4fcc-8f7f-9d1fd90011af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:32.969746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eca882-f848-4fcc-8f7f-9d1fd90011af None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:32.979598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 102/204] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:32.985488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c0d14-3ac0-4a43-a3a0-8413d05a722d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:33.042148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:33.095768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c0d14-3ac0-4a43-a3a0-8413d05a722d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:33.137883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbd4e2-97e9-4bcf-bfcd-2cfb17678e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:33.138530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbd4e2-97e9-4bcf-bfcd-2cfb17678e64 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:33.138821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbd4e2-97e9-4bcf-bfcd-2cfb17678e64 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:33.139102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbd4e2-97e9-4bcf-bfcd-2cfb17678e64 None admin] PATH_INFO: `/v3/users/586479a9c9ed4428b2f6c9bde72351ba` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:33.140310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbd4e2-97e9-4bcf-bfcd-2cfb17678e64 None admin] RBAC: Authorizing `identity:get_user(user_id=586479a9c9ed4428b2f6c9bde72351ba)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:33.141927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbd4e2-97e9-4bcf-bfcd-2cfb17678e64 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:33.143524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 103/205] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:32 2022] GET /identity/v3/users/586479a9c9ed4428b2f6c9bde72351ba => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:33.149697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eca882-f848-4fcc-8f7f-9d1fd90011af None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:33.249930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:33.284068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eca882-f848-4fcc-8f7f-9d1fd90011af None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:33.323853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1701a-0c18-4310-a027-88f83ae2a4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:33.324475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1701a-0c18-4310-a027-88f83ae2a4e7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:33.324755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1701a-0c18-4310-a027-88f83ae2a4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:33.325444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1701a-0c18-4310-a027-88f83ae2a4e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:33.411190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1701a-0c18-4310-a027-88f83ae2a4e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:33.412675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1701a-0c18-4310-a027-88f83ae2a4e7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:33.491820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:33.528067 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 103/206] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:33.533863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbd4e2-97e9-4bcf-bfcd-2cfb17678e64 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:33.676828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbd4e2-97e9-4bcf-bfcd-2cfb17678e64 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:33.694546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:33.716100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f7386-0445-40b3-82bb-deaad6b30851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:33.718181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f7386-0445-40b3-82bb-deaad6b30851 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:33.718181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f7386-0445-40b3-82bb-deaad6b30851 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:33.718181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f7386-0445-40b3-82bb-deaad6b30851 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:33.718718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f7386-0445-40b3-82bb-deaad6b30851 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:33.720326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f7386-0445-40b3-82bb-deaad6b30851 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:33.722217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 104/207] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:33 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:33.728673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1701a-0c18-4310-a027-88f83ae2a4e7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:33.841134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1701a-0c18-4310-a027-88f83ae2a4e7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:33.881878 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e1864-9561-46a5-9916-d1020bb4853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:33.882491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e1864-9561-46a5-9916-d1020bb4853c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:33.882876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e1864-9561-46a5-9916-d1020bb4853c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:33.883842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e1864-9561-46a5-9916-d1020bb4853c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:33.884556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e1864-9561-46a5-9916-d1020bb4853c None admin] RBAC: Adding query filter params (role.id=1714afd0de144939a3022b0632f03aa0, scope.project.id=c59969df6296450f916add4011ddf885, user.id=586479a9c9ed4428b2f6c9bde72351ba) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:33.884869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e1864-9561-46a5-9916-d1020bb4853c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:33.886381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e1864-9561-46a5-9916-d1020bb4853c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:33.897523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 104/208] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:33 2022] GET /identity/v3/role_assignments?user.id=586479a9c9ed4428b2f6c9bde72351ba&scope.project.id=c59969df6296450f916add4011ddf885&role.id=1714afd0de144939a3022b0632f03aa0 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:34.447871 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dec64-0170-4d2d-83ec-5e5e7d28d140 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:34.448255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dec64-0170-4d2d-83ec-5e5e7d28d140 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:34.448424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dec64-0170-4d2d-83ec-5e5e7d28d140 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:34.450039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 105/209] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:34.456197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683efd1-4bb5-4d44-8495-4d128d1c0452 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:34.456197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683efd1-4bb5-4d44-8495-4d128d1c0452 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:34.456197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683efd1-4bb5-4d44-8495-4d128d1c0452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:34.482928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-9683efd1-4bb5-4d44-8495-4d128d1c0452 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:34.484465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9683efd1-4bb5-4d44-8495-4d128d1c0452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:34.533006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:34.581499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 105/210] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:32:34.637970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9658e-a1ea-4481-9a16-de0cf1ab3165 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:34.638255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9658e-a1ea-4481-9a16-de0cf1ab3165 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:34.638703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9658e-a1ea-4481-9a16-de0cf1ab3165 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:34.640520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 106/211] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:34.645494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ead40f-d588-4554-bfde-1d73cedd6b24 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:34.646221 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ead40f-d588-4554-bfde-1d73cedd6b24 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:34.646643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ead40f-d588-4554-bfde-1d73cedd6b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:34.666402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a9ead40f-d588-4554-bfde-1d73cedd6b24 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:34.667844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a9ead40f-d588-4554-bfde-1d73cedd6b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:34.738604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 106/212] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:34.864803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9658e-a1ea-4481-9a16-de0cf1ab3165 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:34.978001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9658e-a1ea-4481-9a16-de0cf1ab3165 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:35.018617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edecff-062b-4529-8d85-3ae67cb75bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:35.019186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edecff-062b-4529-8d85-3ae67cb75bbf None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.019465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edecff-062b-4529-8d85-3ae67cb75bbf None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.019737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edecff-062b-4529-8d85-3ae67cb75bbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.033427 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edecff-062b-4529-8d85-3ae67cb75bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-66edecff-062b-4529-8d85-3ae67cb75bbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:35.038702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:35.039893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:35.040684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:35.041341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ead40f-d588-4554-bfde-1d73cedd6b24 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:35.041745 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 107/213] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:32:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:35.053102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edecff-062b-4529-8d85-3ae67cb75bbf None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:35.113993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:35.154744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:35.159246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ead40f-d588-4554-bfde-1d73cedd6b24 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:35.175904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edecff-062b-4529-8d85-3ae67cb75bbf None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:35.194898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cf1c4-0665-4385-8be0-13de4a9d106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:35.195674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cf1c4-0665-4385-8be0-13de4a9d106c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.195674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cf1c4-0665-4385-8be0-13de4a9d106c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.196229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cf1c4-0665-4385-8be0-13de4a9d106c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.206359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6cf1c4-0665-4385-8be0-13de4a9d106c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:35.209610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2cc5e-1f3e-498f-9d78-9c52a85a4581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:35.210212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2cc5e-1f3e-498f-9d78-9c52a85a4581 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.210490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2cc5e-1f3e-498f-9d78-9c52a85a4581 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.210761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2cc5e-1f3e-498f-9d78-9c52a85a4581 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-2b6cf1c4-0665-4385-8be0-13de4a9d106c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:35.211283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:35.212458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:35.213697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:35.214306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2cc5e-1f3e-498f-9d78-9c52a85a4581 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:35.214306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2cc5e-1f3e-498f-9d78-9c52a85a4581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:35.214436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 107/214] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:32:35.214436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf46712-8216-402a-99fd-db76c7f4be95 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.214436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf46712-8216-402a-99fd-db76c7f4be95 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.214436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf46712-8216-402a-99fd-db76c7f4be95 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.215307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 108/215] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:35.216328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d6ef-56f1-47da-a727-67831369f70b None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.216612 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d6ef-56f1-47da-a727-67831369f70b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.216901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d6ef-56f1-47da-a727-67831369f70b None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.218194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2cc5e-1f3e-498f-9d78-9c52a85a4581 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:35.218371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 109/216] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:35.220591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0d6ef-56f1-47da-a727-67831369f70b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:35.232678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 108/217] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:35.232678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed90d1-1427-4d90-a7c3-fe256d6168fa None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.232678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed90d1-1427-4d90-a7c3-fe256d6168fa None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.232678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed90d1-1427-4d90-a7c3-fe256d6168fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.248598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2fed90d1-1427-4d90-a7c3-fe256d6168fa None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:35.249809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2fed90d1-1427-4d90-a7c3-fe256d6168fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:35.316869 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 109/218] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:35.322165 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0d6ef-56f1-47da-a727-67831369f70b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:35.322545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9de2ce-88a0-4614-90f4-e4b027a51047 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.322831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9de2ce-88a0-4614-90f4-e4b027a51047 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.323104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9de2ce-88a0-4614-90f4-e4b027a51047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.346168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2b9de2ce-88a0-4614-90f4-e4b027a51047 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:35.347367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2b9de2ce-88a0-4614-90f4-e4b027a51047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:35.358329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000ebba-f545-4dd5-a0a4-6deb9dc58a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:35.358944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000ebba-f545-4dd5-a0a4-6deb9dc58a13 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.359240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000ebba-f545-4dd5-a0a4-6deb9dc58a13 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.359490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000ebba-f545-4dd5-a0a4-6deb9dc58a13 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.360153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000ebba-f545-4dd5-a0a4-6deb9dc58a13 None admin] RBAC: Adding query filter params (name=member) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:35.360452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000ebba-f545-4dd5-a0a4-6deb9dc58a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:35.361849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000ebba-f545-4dd5-a0a4-6deb9dc58a13 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:35.370174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 110/219] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:35.370775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000ebba-f545-4dd5-a0a4-6deb9dc58a13 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:35.384221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:35.393507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:35.424710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 110/220] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:35.425677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9de2ce-88a0-4614-90f4-e4b027a51047 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:35.495005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000ebba-f545-4dd5-a0a4-6deb9dc58a13 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:35.535622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c7f61-6f71-4260-9c60-77ace14dc9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:35.536281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c7f61-6f71-4260-9c60-77ace14dc9f8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.536807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c7f61-6f71-4260-9c60-77ace14dc9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.537115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703c7f61-6f71-4260-9c60-77ace14dc9f8 None admin] PATH_INFO: `/v3/users/f660125dad194f68bde4d7d3dad65811` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.538315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c7f61-6f71-4260-9c60-77ace14dc9f8 None admin] RBAC: Authorizing `identity:get_user(user_id=f660125dad194f68bde4d7d3dad65811)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:35.539843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9de2ce-88a0-4614-90f4-e4b027a51047 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:35.540822 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703c7f61-6f71-4260-9c60-77ace14dc9f8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:35.543330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 111/221] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:35 2022] GET /identity/v3/users/f660125dad194f68bde4d7d3dad65811 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:35.550008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c7f61-6f71-4260-9c60-77ace14dc9f8 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:35.586589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a8ae9-2722-4474-8e07-d27476258e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:35.588358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a8ae9-2722-4474-8e07-d27476258e2c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.588797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a8ae9-2722-4474-8e07-d27476258e2c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.589257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a8ae9-2722-4474-8e07-d27476258e2c None admin] PATH_INFO: `/v3/users/a7af7abd2ae149a79ed5d31ed7d75422` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.609890 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a8ae9-2722-4474-8e07-d27476258e2c None admin] RBAC: Authorizing `identity:get_user(user_id=a7af7abd2ae149a79ed5d31ed7d75422)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:35.611363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a8ae9-2722-4474-8e07-d27476258e2c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:35.617192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 111/222] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:35 2022] GET /identity/v3/users/a7af7abd2ae149a79ed5d31ed7d75422 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:35.619320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a8ae9-2722-4474-8e07-d27476258e2c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:35.648204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:35.652629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:35.666688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703c7f61-6f71-4260-9c60-77ace14dc9f8 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:35.705456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d048a7-dfe1-4ba1-949e-21e1c4237230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:35.706289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d048a7-dfe1-4ba1-949e-21e1c4237230 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.706707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d048a7-dfe1-4ba1-949e-21e1c4237230 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.707231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d048a7-dfe1-4ba1-949e-21e1c4237230 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.709157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d048a7-dfe1-4ba1-949e-21e1c4237230 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:35.710874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d048a7-dfe1-4ba1-949e-21e1c4237230 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:35.714279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 112/223] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:35 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:35.715284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d048a7-dfe1-4ba1-949e-21e1c4237230 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:35.726073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a8ae9-2722-4474-8e07-d27476258e2c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:35.760908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5723154-59e2-44f0-bba2-e43aeec1b8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:35.761575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5723154-59e2-44f0-bba2-e43aeec1b8f2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.761858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5723154-59e2-44f0-bba2-e43aeec1b8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.762139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5723154-59e2-44f0-bba2-e43aeec1b8f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.763247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5723154-59e2-44f0-bba2-e43aeec1b8f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:35.764702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5723154-59e2-44f0-bba2-e43aeec1b8f2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:35.766308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 112/224] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:35.766308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5723154-59e2-44f0-bba2-e43aeec1b8f2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:35.815175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d048a7-dfe1-4ba1-949e-21e1c4237230 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:35.850757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a5d6b-a17a-4d78-83d9-26466301c026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:35.851300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a5d6b-a17a-4d78-83d9-26466301c026 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.851585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a5d6b-a17a-4d78-83d9-26466301c026 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.853011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a5d6b-a17a-4d78-83d9-26466301c026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.862420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:35.868998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5723154-59e2-44f0-bba2-e43aeec1b8f2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:35.877418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:35.903241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a324c3-3c5d-494d-894d-896e5b9367a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:35.903861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a324c3-3c5d-494d-894d-896e5b9367a4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:35.904137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a324c3-3c5d-494d-894d-896e5b9367a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:35.904408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a324c3-3c5d-494d-894d-896e5b9367a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:35.914603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a324c3-3c5d-494d-894d-896e5b9367a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-28a324c3-3c5d-494d-894d-896e5b9367a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:35.919407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:35.920345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:35.921195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:35.922456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a5d6b-a17a-4d78-83d9-26466301c026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:35.922543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 113/225] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:32:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:35.922543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a324c3-3c5d-494d-894d-896e5b9367a4 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:35.922695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a5d6b-a17a-4d78-83d9-26466301c026 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:36.020960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a324c3-3c5d-494d-894d-896e5b9367a4 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:36.021281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 113/226] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:36.026110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a5d6b-a17a-4d78-83d9-26466301c026 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:36.059347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496ffa9-69d5-4bfa-9c5b-eb5049de151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:36.060002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ffa9-69d5-4bfa-9c5b-eb5049de151f None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:36.060670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ffa9-69d5-4bfa-9c5b-eb5049de151f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:36.060950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ffa9-69d5-4bfa-9c5b-eb5049de151f None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/users/f660125dad194f68bde4d7d3dad65811/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:36.063221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496ffa9-69d5-4bfa-9c5b-eb5049de151f None admin] RBAC: Authorizing `identity:create_grant(project_id=9ef8ff3249dd4e84ad8692c06a441377, user_id=f660125dad194f68bde4d7d3dad65811, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:36.065033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496ffa9-69d5-4bfa-9c5b-eb5049de151f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:36.075983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:36.119765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a5d6b-a17a-4d78-83d9-26466301c026 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:36.138330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 114/227] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:32:35 2022] PUT /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/users/f660125dad194f68bde4d7d3dad65811/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:32:36.139259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496ffa9-69d5-4bfa-9c5b-eb5049de151f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:36.158969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:36.160734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fba05-cc71-48ce-a9de-1930b5b7c250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:36.161409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fba05-cc71-48ce-a9de-1930b5b7c250 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:36.161692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fba05-cc71-48ce-a9de-1930b5b7c250 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:36.161965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fba05-cc71-48ce-a9de-1930b5b7c250 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:36.162647 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fba05-cc71-48ce-a9de-1930b5b7c250 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:36.164127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fba05-cc71-48ce-a9de-1930b5b7c250 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:36.261847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496ffa9-69d5-4bfa-9c5b-eb5049de151f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:36.262869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 114/228] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:32:36 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:36.263772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fba05-cc71-48ce-a9de-1930b5b7c250 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:36.316462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c684e-1191-4d3e-afac-dbb873f880cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:36.317196 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c684e-1191-4d3e-afac-dbb873f880cc None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:36.317488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c684e-1191-4d3e-afac-dbb873f880cc None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:36.317767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c684e-1191-4d3e-afac-dbb873f880cc None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:36.318496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c684e-1191-4d3e-afac-dbb873f880cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:36.318809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c684e-1191-4d3e-afac-dbb873f880cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:36.320593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c684e-1191-4d3e-afac-dbb873f880cc None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:36.331497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 115/229] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:32:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:36.339126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c684e-1191-4d3e-afac-dbb873f880cc None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:36.406187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fba05-cc71-48ce-a9de-1930b5b7c250 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:36.425783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:36.432973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:36.457338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869bc48-7205-4ec0-9721-414dfd76ee35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:36.458338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869bc48-7205-4ec0-9721-414dfd76ee35 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:36.458840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869bc48-7205-4ec0-9721-414dfd76ee35 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:36.459313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869bc48-7205-4ec0-9721-414dfd76ee35 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:36.459431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c684e-1191-4d3e-afac-dbb873f880cc None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:36.461023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869bc48-7205-4ec0-9721-414dfd76ee35 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:36.463663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869bc48-7205-4ec0-9721-414dfd76ee35 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:36.465733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 115/230] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:36 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:36.472927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869bc48-7205-4ec0-9721-414dfd76ee35 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:36.504626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada4a3e-87ec-48a5-a10a-bd5f7aa4c0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:36.505747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada4a3e-87ec-48a5-a10a-bd5f7aa4c0b5 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:36.506351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada4a3e-87ec-48a5-a10a-bd5f7aa4c0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:36.506634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada4a3e-87ec-48a5-a10a-bd5f7aa4c0b5 None admin] PATH_INFO: `/v3/users/f660125dad194f68bde4d7d3dad65811` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:36.508282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada4a3e-87ec-48a5-a10a-bd5f7aa4c0b5 None admin] RBAC: Authorizing `identity:get_user(user_id=f660125dad194f68bde4d7d3dad65811)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:36.509584 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada4a3e-87ec-48a5-a10a-bd5f7aa4c0b5 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:36.511253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 116/231] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:36 2022] GET /identity/v3/users/f660125dad194f68bde4d7d3dad65811 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:36.517129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada4a3e-87ec-48a5-a10a-bd5f7aa4c0b5 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:36.596282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869bc48-7205-4ec0-9721-414dfd76ee35 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:36.633981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada4a3e-87ec-48a5-a10a-bd5f7aa4c0b5 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:36.644607 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da507597-baaa-4efa-861f-2a8249484a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:36.645258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da507597-baaa-4efa-861f-2a8249484a66 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:36.645537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da507597-baaa-4efa-861f-2a8249484a66 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:36.645813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da507597-baaa-4efa-861f-2a8249484a66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:36.646584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da507597-baaa-4efa-861f-2a8249484a66 None admin] RBAC: Adding query filter params (role.id=eccc3f8721b1466b8a1968b77e5c464f, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377, user.id=a7af7abd2ae149a79ed5d31ed7d75422) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:36.646892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da507597-baaa-4efa-861f-2a8249484a66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:36.648341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da507597-baaa-4efa-861f-2a8249484a66 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:36.656674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 116/232] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:36 2022] GET /identity/v3/role_assignments?user.id=a7af7abd2ae149a79ed5d31ed7d75422&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:36.687476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d3cba-be9b-41b9-8611-cefeb9e1a340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:36.688175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d3cba-be9b-41b9-8611-cefeb9e1a340 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:36.688461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d3cba-be9b-41b9-8611-cefeb9e1a340 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:36.688739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d3cba-be9b-41b9-8611-cefeb9e1a340 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:36.690061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d3cba-be9b-41b9-8611-cefeb9e1a340 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:36.691609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d3cba-be9b-41b9-8611-cefeb9e1a340 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:36.693560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 117/233] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:36 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:36.700140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da507597-baaa-4efa-861f-2a8249484a66 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:36.755470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:36.819476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da507597-baaa-4efa-861f-2a8249484a66 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:36.859803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb9a14-8d24-44ee-8e58-cfcd3b5cdd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:36.860487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb9a14-8d24-44ee-8e58-cfcd3b5cdd01 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:36.860768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb9a14-8d24-44ee-8e58-cfcd3b5cdd01 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:36.861064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb9a14-8d24-44ee-8e58-cfcd3b5cdd01 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/users/f660125dad194f68bde4d7d3dad65811/roles/6b3a5d54d6044e9b8d8270e7ff3145e8` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:36.863404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb9a14-8d24-44ee-8e58-cfcd3b5cdd01 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ef8ff3249dd4e84ad8692c06a441377, user_id=f660125dad194f68bde4d7d3dad65811, role_id=6b3a5d54d6044e9b8d8270e7ff3145e8)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:36.864864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb9a14-8d24-44ee-8e58-cfcd3b5cdd01 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:36.945023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 117/234] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:32:36 2022] PUT /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/users/f660125dad194f68bde4d7d3dad65811/roles/6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:32:38.741161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71fb90-84d6-49ef-889d-6826a54dbe14 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:38.741696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71fb90-84d6-49ef-889d-6826a54dbe14 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:38.741696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71fb90-84d6-49ef-889d-6826a54dbe14 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:38.744078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 118/235] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:38.749412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaf95b-6e50-4401-9158-df7a3d5c535f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:38.749827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaf95b-6e50-4401-9158-df7a3d5c535f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:38.750020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaf95b-6e50-4401-9158-df7a3d5c535f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:38.778063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-3adaf95b-6e50-4401-9158-df7a3d5c535f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:38.779283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3adaf95b-6e50-4401-9158-df7a3d5c535f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:38.866454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 118/236] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:39.206116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71fb90-84d6-49ef-889d-6826a54dbe14 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:39.225181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:39.282790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00d5cc-51c9-48fd-8fde-23ccbf08544d None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.283138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00d5cc-51c9-48fd-8fde-23ccbf08544d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.283360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00d5cc-51c9-48fd-8fde-23ccbf08544d None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.285174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 119/237] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:39.291609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8aadf2-dc3f-41ce-8697-6f016be024e0 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.291894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8aadf2-dc3f-41ce-8697-6f016be024e0 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.292161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8aadf2-dc3f-41ce-8697-6f016be024e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.312778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-ee8aadf2-dc3f-41ce-8697-6f016be024e0 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:39.314022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ee8aadf2-dc3f-41ce-8697-6f016be024e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:39.345188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:39.359803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71fb90-84d6-49ef-889d-6826a54dbe14 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:39.399451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 120/238] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:32:39.401443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631061c6-9bbc-47fe-b8f9-fd4797785b6a None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.401780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631061c6-9bbc-47fe-b8f9-fd4797785b6a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.402058 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631061c6-9bbc-47fe-b8f9-fd4797785b6a None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.403473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 121/239] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:39.404444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9bcdb-9234-4b34-aef0-bb5d0dd80497 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.404703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9bcdb-9234-4b34-aef0-bb5d0dd80497 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.404974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9bcdb-9234-4b34-aef0-bb5d0dd80497 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.409157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 122/240] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:39.411747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6a268-d6a5-40af-9867-1dda74469109 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.412038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6a268-d6a5-40af-9867-1dda74469109 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.412320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6a268-d6a5-40af-9867-1dda74469109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.416305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb1e85-d8c2-4591-9fd4-cb2bc8d13b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:39.417205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1e85-d8c2-4591-9fd4-cb2bc8d13b0e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.417703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1e85-d8c2-4591-9fd4-cb2bc8d13b0e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.418181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1e85-d8c2-4591-9fd4-cb2bc8d13b0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.430388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb1e85-d8c2-4591-9fd4-cb2bc8d13b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-abdb1e85-d8c2-4591-9fd4-cb2bc8d13b0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:39.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:39.436455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:39.437284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:39.437888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-22a6a268-d6a5-40af-9867-1dda74469109 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:39.437888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-22a6a268-d6a5-40af-9867-1dda74469109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:39.440124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 119/241] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:32:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:39.441643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b92d1-58fa-4bd9-a468-0c048a9d98dd None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.442130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b92d1-58fa-4bd9-a468-0c048a9d98dd None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.442571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b92d1-58fa-4bd9-a468-0c048a9d98dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.462305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-090b92d1-58fa-4bd9-a468-0c048a9d98dd None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:39.463733 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-090b92d1-58fa-4bd9-a468-0c048a9d98dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:39.527892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 123/242] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:39.533234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6a268-d6a5-40af-9867-1dda74469109 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:39.549980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 120/243] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:39.634683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:39.641935 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6a268-d6a5-40af-9867-1dda74469109 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:39.681774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f8511-2c39-482f-9d2a-d59aa3fca6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:39.682403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f8511-2c39-482f-9d2a-d59aa3fca6ca None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.682685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f8511-2c39-482f-9d2a-d59aa3fca6ca None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.682959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f8511-2c39-482f-9d2a-d59aa3fca6ca None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.683630 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f8511-2c39-482f-9d2a-d59aa3fca6ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:39.683939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f8511-2c39-482f-9d2a-d59aa3fca6ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:39.685336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f8511-2c39-482f-9d2a-d59aa3fca6ca None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:39.698338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 124/244] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:39.703993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b92d1-58fa-4bd9-a468-0c048a9d98dd None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:39.706857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:39.732730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f8511-2c39-482f-9d2a-d59aa3fca6ca None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:39.879449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b92d1-58fa-4bd9-a468-0c048a9d98dd None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:39.900818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f8511-2c39-482f-9d2a-d59aa3fca6ca None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:39.917386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114351a1-0985-49a5-8fe2-588fdc523f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:39.918031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114351a1-0985-49a5-8fe2-588fdc523f7a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.918312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114351a1-0985-49a5-8fe2-588fdc523f7a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.918655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114351a1-0985-49a5-8fe2-588fdc523f7a None admin] PATH_INFO: `/v3/users/f660125dad194f68bde4d7d3dad65811` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.920529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114351a1-0985-49a5-8fe2-588fdc523f7a None admin] RBAC: Authorizing `identity:get_user(user_id=f660125dad194f68bde4d7d3dad65811)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:39.922066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114351a1-0985-49a5-8fe2-588fdc523f7a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:39.923674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 121/245] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:39 2022] GET /identity/v3/users/f660125dad194f68bde4d7d3dad65811 => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:39.926694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5da3a-918e-46a3-8728-4f989241135b None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.926983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5da3a-918e-46a3-8728-4f989241135b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.927260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5da3a-918e-46a3-8728-4f989241135b None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.929097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 122/246] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:39.930897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5da3a-918e-46a3-8728-4f989241135b None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:39.944532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914ecb2-adaf-4d78-bed6-1cc610f1dffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:39.945271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914ecb2-adaf-4d78-bed6-1cc610f1dffc None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:39.945624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914ecb2-adaf-4d78-bed6-1cc610f1dffc None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:39.946342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914ecb2-adaf-4d78-bed6-1cc610f1dffc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:39.957253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914ecb2-adaf-4d78-bed6-1cc610f1dffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-7914ecb2-adaf-4d78-bed6-1cc610f1dffc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:39.962719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:39.963721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:39.964981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 125/247] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:32:39.969412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914ecb2-adaf-4d78-bed6-1cc610f1dffc None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:39.987579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:40.004785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:40.036946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5da3a-918e-46a3-8728-4f989241135b None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:40.071770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f8fc3-47d4-4d5a-b7d9-1b5151cc2500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:40.072405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f8fc3-47d4-4d5a-b7d9-1b5151cc2500 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.072648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914ecb2-adaf-4d78-bed6-1cc610f1dffc None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:40.072749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f8fc3-47d4-4d5a-b7d9-1b5151cc2500 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.072953 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f8fc3-47d4-4d5a-b7d9-1b5151cc2500 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.083244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f8fc3-47d4-4d5a-b7d9-1b5151cc2500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-145f8fc3-47d4-4d5a-b7d9-1b5151cc2500 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:40.088667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:40.089757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:40.090724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 123/248] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:40.091528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783fd7e-9415-4add-98d6-d2fecef944e4 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.091528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783fd7e-9415-4add-98d6-d2fecef944e4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.091528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783fd7e-9415-4add-98d6-d2fecef944e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.111288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c783fd7e-9415-4add-98d6-d2fecef944e4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:40.112501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c783fd7e-9415-4add-98d6-d2fecef944e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:40.114741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629c27b-d83b-470d-8480-0a85dd2419a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:40.115359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629c27b-d83b-470d-8480-0a85dd2419a2 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.115633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629c27b-d83b-470d-8480-0a85dd2419a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.115910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629c27b-d83b-470d-8480-0a85dd2419a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.179435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 124/249] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:40.180900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783fd7e-9415-4add-98d6-d2fecef944e4 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:40.195082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629c27b-d83b-470d-8480-0a85dd2419a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:40.197627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629c27b-d83b-470d-8480-0a85dd2419a2 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:40.216079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:40.261500 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:40.304231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783fd7e-9415-4add-98d6-d2fecef944e4 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:40.305808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 126/250] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:40.310585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629c27b-d83b-470d-8480-0a85dd2419a2 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:40.345448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8705821-0ffc-43ef-94be-b30cb24124dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:40.346082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8705821-0ffc-43ef-94be-b30cb24124dc None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.346363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8705821-0ffc-43ef-94be-b30cb24124dc None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.346638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8705821-0ffc-43ef-94be-b30cb24124dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.347748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8705821-0ffc-43ef-94be-b30cb24124dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:40.349238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8705821-0ffc-43ef-94be-b30cb24124dc None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:40.353087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 125/251] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:40.353087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8705821-0ffc-43ef-94be-b30cb24124dc None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:40.417277 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629c27b-d83b-470d-8480-0a85dd2419a2 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:40.434756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:40.454796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e6b44-88a2-4390-be56-a59aaa95b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:40.455411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e6b44-88a2-4390-be56-a59aaa95b1af None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.455691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e6b44-88a2-4390-be56-a59aaa95b1af None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.455966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e6b44-88a2-4390-be56-a59aaa95b1af None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.456626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e6b44-88a2-4390-be56-a59aaa95b1af None admin] RBAC: Adding query filter params (name=reader) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:40.456927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e6b44-88a2-4390-be56-a59aaa95b1af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:40.458330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e6b44-88a2-4390-be56-a59aaa95b1af None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:40.462351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8705821-0ffc-43ef-94be-b30cb24124dc None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:40.467882 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 127/252] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:40.468708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e6b44-88a2-4390-be56-a59aaa95b1af None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:40.489401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:40.497597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c06859-13ba-4373-b444-a34127a24b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:40.498210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c06859-13ba-4373-b444-a34127a24b2c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.498486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c06859-13ba-4373-b444-a34127a24b2c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.503546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c06859-13ba-4373-b444-a34127a24b2c None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.503546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c06859-13ba-4373-b444-a34127a24b2c None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:40.503546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c06859-13ba-4373-b444-a34127a24b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:40.503546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c06859-13ba-4373-b444-a34127a24b2c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:40.509463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 126/253] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:40.513258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c06859-13ba-4373-b444-a34127a24b2c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:40.580286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e6b44-88a2-4390-be56-a59aaa95b1af None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:40.616584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307560-2d34-487e-8f6a-fdc44bf812aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:40.617877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307560-2d34-487e-8f6a-fdc44bf812aa None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.618270 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307560-2d34-487e-8f6a-fdc44bf812aa None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.618554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307560-2d34-487e-8f6a-fdc44bf812aa None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.619696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6307560-2d34-487e-8f6a-fdc44bf812aa None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:40.619799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c06859-13ba-4373-b444-a34127a24b2c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:40.621175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6307560-2d34-487e-8f6a-fdc44bf812aa None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:40.622824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 128/254] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:40 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:40.623596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307560-2d34-487e-8f6a-fdc44bf812aa None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:40.653964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c21829-a06c-469e-99e0-ee4aa7c1f8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:40.654573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c21829-a06c-469e-99e0-ee4aa7c1f8b4 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.654856 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c21829-a06c-469e-99e0-ee4aa7c1f8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.655131 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c21829-a06c-469e-99e0-ee4aa7c1f8b4 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.655774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c21829-a06c-469e-99e0-ee4aa7c1f8b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:40.657255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c21829-a06c-469e-99e0-ee4aa7c1f8b4 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:40.669803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:40.721902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307560-2d34-487e-8f6a-fdc44bf812aa None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:40.725824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 127/255] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:32:40 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:40.732532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c21829-a06c-469e-99e0-ee4aa7c1f8b4 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:40.762978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb7c2e-ef72-4bc5-bca3-a61e9bd8190f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:40.763859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb7c2e-ef72-4bc5-bca3-a61e9bd8190f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.764187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb7c2e-ef72-4bc5-bca3-a61e9bd8190f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.764631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb7c2e-ef72-4bc5-bca3-a61e9bd8190f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.777929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb7c2e-ef72-4bc5-bca3-a61e9bd8190f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-abdb7c2e-ef72-4bc5-bca3-a61e9bd8190f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:40.783986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:40.785180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:40.786164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:40.787526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 129/256] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:32:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:40.787526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb7c2e-ef72-4bc5-bca3-a61e9bd8190f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:40.800392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:40.865247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c21829-a06c-469e-99e0-ee4aa7c1f8b4 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:40.912669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c89a5-4e9d-47ea-bb98-4a91cbc19fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:40.913366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c89a5-4e9d-47ea-bb98-4a91cbc19fbf None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.913635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c89a5-4e9d-47ea-bb98-4a91cbc19fbf None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.913910 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c89a5-4e9d-47ea-bb98-4a91cbc19fbf None admin] PATH_INFO: `/v3/users/edc74aeeeadb4d50a50d3b1a5985e4fd` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.919680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb7c2e-ef72-4bc5-bca3-a61e9bd8190f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:40.927995 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:40.931700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c89a5-4e9d-47ea-bb98-4a91cbc19fbf None admin] RBAC: Authorizing `identity:get_user(user_id=edc74aeeeadb4d50a50d3b1a5985e4fd)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:40.933274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c89a5-4e9d-47ea-bb98-4a91cbc19fbf None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:40.935873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 128/257] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:40 2022] GET /identity/v3/users/edc74aeeeadb4d50a50d3b1a5985e4fd => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:40.935873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c89a5-4e9d-47ea-bb98-4a91cbc19fbf None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:40.965230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcdf14-b349-43c1-a452-f9b223af95db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:40.965917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcdf14-b349-43c1-a452-f9b223af95db None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:40.966194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcdf14-b349-43c1-a452-f9b223af95db None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:40.966467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcdf14-b349-43c1-a452-f9b223af95db None admin] PATH_INFO: `/v3/users/a7af7abd2ae149a79ed5d31ed7d75422` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:40.967992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bcdf14-b349-43c1-a452-f9b223af95db None admin] RBAC: Authorizing `identity:get_user(user_id=a7af7abd2ae149a79ed5d31ed7d75422)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:40.969699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bcdf14-b349-43c1-a452-f9b223af95db None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:40.972533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 130/258] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:40 2022] GET /identity/v3/users/a7af7abd2ae149a79ed5d31ed7d75422 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:40.973502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcdf14-b349-43c1-a452-f9b223af95db None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:41.069128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c89a5-4e9d-47ea-bb98-4a91cbc19fbf None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:41.085720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:41.100222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcdf14-b349-43c1-a452-f9b223af95db None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:41.112053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db878de-449a-48e3-a377-9df56f9bd83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:41.113143 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db878de-449a-48e3-a377-9df56f9bd83b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:41.113581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db878de-449a-48e3-a377-9df56f9bd83b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:41.113863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db878de-449a-48e3-a377-9df56f9bd83b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:41.114604 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db878de-449a-48e3-a377-9df56f9bd83b None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377, user.id=f660125dad194f68bde4d7d3dad65811) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:41.114917 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db878de-449a-48e3-a377-9df56f9bd83b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:41.116947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db878de-449a-48e3-a377-9df56f9bd83b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:41.125030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 129/259] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:40 2022] GET /identity/v3/role_assignments?user.id=f660125dad194f68bde4d7d3dad65811&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=5c777de40b8449329d714d8c2229115e => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:41.130210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db878de-449a-48e3-a377-9df56f9bd83b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:41.137856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccb1e9-2ee4-42a1-b3a1-c065f9966b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:41.139068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccb1e9-2ee4-42a1-b3a1-c065f9966b32 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:41.139068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccb1e9-2ee4-42a1-b3a1-c065f9966b32 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:41.139068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccb1e9-2ee4-42a1-b3a1-c065f9966b32 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:41.139775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccb1e9-2ee4-42a1-b3a1-c065f9966b32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:41.140135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccb1e9-2ee4-42a1-b3a1-c065f9966b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:41.141656 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccb1e9-2ee4-42a1-b3a1-c065f9966b32 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:41.152021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 131/260] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:32:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:41.152974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccb1e9-2ee4-42a1-b3a1-c065f9966b32 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:41.198636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:41.273214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccb1e9-2ee4-42a1-b3a1-c065f9966b32 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:41.311643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79915bd4-4cae-4c6d-9f67-b10c6d858547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:41.312596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79915bd4-4cae-4c6d-9f67-b10c6d858547 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:41.312882 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79915bd4-4cae-4c6d-9f67-b10c6d858547 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:41.313198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79915bd4-4cae-4c6d-9f67-b10c6d858547 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:41.314459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79915bd4-4cae-4c6d-9f67-b10c6d858547 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:41.315951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79915bd4-4cae-4c6d-9f67-b10c6d858547 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:41.318077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 132/261] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:41 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:41.318934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79915bd4-4cae-4c6d-9f67-b10c6d858547 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:41.393098 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db878de-449a-48e3-a377-9df56f9bd83b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:41.441682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79915bd4-4cae-4c6d-9f67-b10c6d858547 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:41.445179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:41.449824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089b212-44d6-4be0-a3a4-3c6a512eaac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:41.450456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089b212-44d6-4be0-a3a4-3c6a512eaac2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:41.450738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089b212-44d6-4be0-a3a4-3c6a512eaac2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:41.451014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089b212-44d6-4be0-a3a4-3c6a512eaac2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:41.492118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a650a-ed21-43a0-9991-ce26062e8c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:41.493861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a650a-ed21-43a0-9991-ce26062e8c62 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:41.494315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a650a-ed21-43a0-9991-ce26062e8c62 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:41.495000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a650a-ed21-43a0-9991-ce26062e8c62 None admin] PATH_INFO: `/v3/users/f660125dad194f68bde4d7d3dad65811` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:41.496349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a650a-ed21-43a0-9991-ce26062e8c62 None admin] RBAC: Authorizing `identity:get_user(user_id=f660125dad194f68bde4d7d3dad65811)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:41.498001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:41.499443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a650a-ed21-43a0-9991-ce26062e8c62 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:41.501231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 133/262] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:41 2022] GET /identity/v3/users/f660125dad194f68bde4d7d3dad65811 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:41.502230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a650a-ed21-43a0-9991-ce26062e8c62 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:41.548965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089b212-44d6-4be0-a3a4-3c6a512eaac2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:41.551308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089b212-44d6-4be0-a3a4-3c6a512eaac2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:41.656563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a650a-ed21-43a0-9991-ce26062e8c62 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:41.688979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 130/263] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:41.690136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089b212-44d6-4be0-a3a4-3c6a512eaac2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:41.698381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64418db-e80e-420f-ad35-abf1c273d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:41.699194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64418db-e80e-420f-ad35-abf1c273d83c None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:41.699583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64418db-e80e-420f-ad35-abf1c273d83c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:41.699965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64418db-e80e-420f-ad35-abf1c273d83c None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/users/a7af7abd2ae149a79ed5d31ed7d75422/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:41.702922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64418db-e80e-420f-ad35-abf1c273d83c None admin] RBAC: Authorizing `identity:create_grant(project_id=9ef8ff3249dd4e84ad8692c06a441377, user_id=a7af7abd2ae149a79ed5d31ed7d75422, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:41.704857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64418db-e80e-420f-ad35-abf1c273d83c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:41.762699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:41.774276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 134/264] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:32:41 2022] PUT /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/users/a7af7abd2ae149a79ed5d31ed7d75422/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:32:41.775531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64418db-e80e-420f-ad35-abf1c273d83c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:41.801709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089b212-44d6-4be0-a3a4-3c6a512eaac2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:41.815943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:41.842789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654ac64-1eaf-4bc6-9b2c-87962129dd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:41.843504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654ac64-1eaf-4bc6-9b2c-87962129dd74 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:41.843783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654ac64-1eaf-4bc6-9b2c-87962129dd74 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:41.844055 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654ac64-1eaf-4bc6-9b2c-87962129dd74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:41.931707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64418db-e80e-420f-ad35-abf1c273d83c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:41.934926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654ac64-1eaf-4bc6-9b2c-87962129dd74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:41.943648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654ac64-1eaf-4bc6-9b2c-87962129dd74 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:41.994582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19676a33-403f-437d-b568-d163354f8c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:41.995285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19676a33-403f-437d-b568-d163354f8c45 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:41.995565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19676a33-403f-437d-b568-d163354f8c45 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:41.995841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19676a33-403f-437d-b568-d163354f8c45 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:41.998274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19676a33-403f-437d-b568-d163354f8c45 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:41.999738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19676a33-403f-437d-b568-d163354f8c45 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:42.001634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 135/265] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:41 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:42.007297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19676a33-403f-437d-b568-d163354f8c45 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:42.059267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:42.103541 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 131/266] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:42.112663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654ac64-1eaf-4bc6-9b2c-87962129dd74 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:42.142889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:42.146474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19676a33-403f-437d-b568-d163354f8c45 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:42.189623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfba35e-14dd-434a-aa27-c18a526b319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:42.190162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfba35e-14dd-434a-aa27-c18a526b319b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:42.190442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfba35e-14dd-434a-aa27-c18a526b319b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:42.190774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfba35e-14dd-434a-aa27-c18a526b319b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:42.191508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfba35e-14dd-434a-aa27-c18a526b319b None admin] RBAC: Adding query filter params (role.id=1714afd0de144939a3022b0632f03aa0, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377, user.id=edc74aeeeadb4d50a50d3b1a5985e4fd) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:42.191817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfba35e-14dd-434a-aa27-c18a526b319b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:42.193286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfba35e-14dd-434a-aa27-c18a526b319b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:42.200055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 136/267] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:42 2022] GET /identity/v3/role_assignments?user.id=edc74aeeeadb4d50a50d3b1a5985e4fd&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=1714afd0de144939a3022b0632f03aa0 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:42.311512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654ac64-1eaf-4bc6-9b2c-87962129dd74 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:42.354558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a2d9e-2cf9-4415-a1b4-96fa529b129a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:42.355227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a2d9e-2cf9-4415-a1b4-96fa529b129a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:42.355510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a2d9e-2cf9-4415-a1b4-96fa529b129a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:42.355786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a2d9e-2cf9-4415-a1b4-96fa529b129a None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:42.356935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a2d9e-2cf9-4415-a1b4-96fa529b129a None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:42.358437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a2d9e-2cf9-4415-a1b4-96fa529b129a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:42.360417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 132/268] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:42 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:42.366441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfba35e-14dd-434a-aa27-c18a526b319b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:42.518794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfba35e-14dd-434a-aa27-c18a526b319b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:42.572746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb067c-6cca-442c-bd4c-9c5b89dab0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:42.575413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb067c-6cca-442c-bd4c-9c5b89dab0bc None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:42.575877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb067c-6cca-442c-bd4c-9c5b89dab0bc None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:42.576326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb067c-6cca-442c-bd4c-9c5b89dab0bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:42.577273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb067c-6cca-442c-bd4c-9c5b89dab0bc None admin] RBAC: Adding query filter params (role.id=6b3a5d54d6044e9b8d8270e7ff3145e8, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377, user.id=f660125dad194f68bde4d7d3dad65811) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:42.577788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb067c-6cca-442c-bd4c-9c5b89dab0bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:42.579480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb067c-6cca-442c-bd4c-9c5b89dab0bc None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:42.585744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 137/269] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:42 2022] GET /identity/v3/role_assignments?user.id=f660125dad194f68bde4d7d3dad65811&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:43.339163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cba7f2-4cd8-474c-bac2-69a2a647f73e None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:43.339739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cba7f2-4cd8-474c-bac2-69a2a647f73e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:43.339739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cba7f2-4cd8-474c-bac2-69a2a647f73e None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:43.341174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 133/270] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:43.347510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb67e1-b79a-4741-8127-d2ef75564eed None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:43.348465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb67e1-b79a-4741-8127-d2ef75564eed None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:43.348969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb67e1-b79a-4741-8127-d2ef75564eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:43.369001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-55fb67e1-b79a-4741-8127-d2ef75564eed None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:43.370561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-55fb67e1-b79a-4741-8127-d2ef75564eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:43.394227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:43.453160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 138/271] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:43.463866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6371b-3203-430b-bbd9-0d00508739a5 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:43.464221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6371b-3203-430b-bbd9-0d00508739a5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:43.464506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6371b-3203-430b-bbd9-0d00508739a5 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:43.466179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 134/272] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:43.471666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f7922-a734-4151-bf20-8b49c30a6026 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:43.471962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f7922-a734-4151-bf20-8b49c30a6026 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:43.472242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f7922-a734-4151-bf20-8b49c30a6026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:43.491188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-645f7922-a734-4151-bf20-8b49c30a6026 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:43.492594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-645f7922-a734-4151-bf20-8b49c30a6026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:43.567032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 139/273] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:43.724271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6371b-3203-430b-bbd9-0d00508739a5 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:43.762608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:43.833405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6371b-3203-430b-bbd9-0d00508739a5 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:43.860381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f7922-a734-4151-bf20-8b49c30a6026 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:43.877082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcdf4c-b183-4397-9e4b-e999fdd6b4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:43.877965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcdf4c-b183-4397-9e4b-e999fdd6b4bb None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:43.878255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcdf4c-b183-4397-9e4b-e999fdd6b4bb None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:43.878534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcdf4c-b183-4397-9e4b-e999fdd6b4bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:43.892452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcdf4c-b183-4397-9e4b-e999fdd6b4bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-aefcdf4c-b183-4397-9e4b-e999fdd6b4bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:43.897971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:43.899414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:43.900365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:43.903123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 135/274] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:43.914078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcdf4c-b183-4397-9e4b-e999fdd6b4bb None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:43.959442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:43.973786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f7922-a734-4151-bf20-8b49c30a6026 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:44.014876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a7129-9a78-4669-8a2a-df891772b698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:44.015642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a7129-9a78-4669-8a2a-df891772b698 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:44.015930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a7129-9a78-4669-8a2a-df891772b698 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:44.016208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a7129-9a78-4669-8a2a-df891772b698 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:44.017405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a7129-9a78-4669-8a2a-df891772b698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:44.019055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a7129-9a78-4669-8a2a-df891772b698 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:44.019678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:44.022258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 140/275] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:44.026729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcdf4c-b183-4397-9e4b-e999fdd6b4bb None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:44.027498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a7129-9a78-4669-8a2a-df891772b698 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:44.072159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247172d-6363-4744-a568-474de31e0383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:44.073279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247172d-6363-4744-a568-474de31e0383 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:44.073730 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247172d-6363-4744-a568-474de31e0383 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:44.074694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247172d-6363-4744-a568-474de31e0383 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:44.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247172d-6363-4744-a568-474de31e0383 None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:44.076032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247172d-6363-4744-a568-474de31e0383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:44.078526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247172d-6363-4744-a568-474de31e0383 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:44.095162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 136/276] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:44.096789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247172d-6363-4744-a568-474de31e0383 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:44.147092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a7129-9a78-4669-8a2a-df891772b698 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:44.187764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aeee97-b701-4338-9eb0-b9b5030bec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:44.188381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aeee97-b701-4338-9eb0-b9b5030bec69 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:44.188660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aeee97-b701-4338-9eb0-b9b5030bec69 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:44.188947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aeee97-b701-4338-9eb0-b9b5030bec69 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:44.190439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aeee97-b701-4338-9eb0-b9b5030bec69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:44.191878 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aeee97-b701-4338-9eb0-b9b5030bec69 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:44.248494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247172d-6363-4744-a568-474de31e0383 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:44.283483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac7d99-6f95-4a39-b588-37475af07cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:44.284111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac7d99-6f95-4a39-b588-37475af07cfa None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:44.284389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac7d99-6f95-4a39-b588-37475af07cfa None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:44.284660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac7d99-6f95-4a39-b588-37475af07cfa None admin] PATH_INFO: `/v3/users/620392eb9b154c35ac6dc5bd40c01edc` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:44.298543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac7d99-6f95-4a39-b588-37475af07cfa None admin] RBAC: Authorizing `identity:get_user(user_id=620392eb9b154c35ac6dc5bd40c01edc)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:44.300036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac7d99-6f95-4a39-b588-37475af07cfa None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:44.307262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 137/277] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:44 2022] GET /identity/v3/users/620392eb9b154c35ac6dc5bd40c01edc => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:44.308709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac7d99-6f95-4a39-b588-37475af07cfa None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:44.315831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 141/278] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:32:44 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:32:44.338562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:44.431966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac7d99-6f95-4a39-b588-37475af07cfa None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:44.432826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e8a0-a530-4e8a-9a88-9d9b80361b78 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:44.433213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e8a0-a530-4e8a-9a88-9d9b80361b78 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:44.433387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e8a0-a530-4e8a-9a88-9d9b80361b78 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:44.434743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 142/279] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:32:44.440374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866482c1-0e1c-469b-b6dd-36033b7a20ae None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:44.440674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866482c1-0e1c-469b-b6dd-36033b7a20ae None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:44.440928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866482c1-0e1c-469b-b6dd-36033b7a20ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:44.462090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-866482c1-0e1c-469b-b6dd-36033b7a20ae None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:44.463554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-866482c1-0e1c-469b-b6dd-36033b7a20ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:44.473813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:44.479413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70ad52-70f3-40f1-a0f7-8d5b6cfe6db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:44.480582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70ad52-70f3-40f1-a0f7-8d5b6cfe6db8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:44.481010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70ad52-70f3-40f1-a0f7-8d5b6cfe6db8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:44.481592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70ad52-70f3-40f1-a0f7-8d5b6cfe6db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:44.482874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70ad52-70f3-40f1-a0f7-8d5b6cfe6db8 None admin] RBAC: Adding query filter params (role.id=eccc3f8721b1466b8a1968b77e5c464f, user.id=620392eb9b154c35ac6dc5bd40c01edc, scope.system=all) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:44.483916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70ad52-70f3-40f1-a0f7-8d5b6cfe6db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:44.485970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70ad52-70f3-40f1-a0f7-8d5b6cfe6db8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:44.493390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 138/280] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 8 18:32:44 2022] GET /identity/v3/role_assignments?user.id=620392eb9b154c35ac6dc5bd40c01edc&scope.system=all&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:44.558174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 143/281] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:44.584674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fe4f1-51c3-42be-a49a-8405c2a45ff2 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:44.584988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fe4f1-51c3-42be-a49a-8405c2a45ff2 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:44.585284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fe4f1-51c3-42be-a49a-8405c2a45ff2 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:44.587409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 139/282] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:44.595569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430ca04-9085-4f3b-bc9c-fb79eed6766b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:44.595852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430ca04-9085-4f3b-bc9c-fb79eed6766b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:44.596132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430ca04-9085-4f3b-bc9c-fb79eed6766b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:44.622868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8430ca04-9085-4f3b-bc9c-fb79eed6766b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:44.624142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8430ca04-9085-4f3b-bc9c-fb79eed6766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:44.699992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 144/283] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:44.841861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fe4f1-51c3-42be-a49a-8405c2a45ff2 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:44.949909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:44.970847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fe4f1-51c3-42be-a49a-8405c2a45ff2 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:45.003063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430ca04-9085-4f3b-bc9c-fb79eed6766b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:45.021610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b9c7-74e4-497f-b0d7-5ba63c09e309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:45.022256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b9c7-74e4-497f-b0d7-5ba63c09e309 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:45.022538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b9c7-74e4-497f-b0d7-5ba63c09e309 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:45.022812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b9c7-74e4-497f-b0d7-5ba63c09e309 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:45.043638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47b9c7-74e4-497f-b0d7-5ba63c09e309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-4a47b9c7-74e4-497f-b0d7-5ba63c09e309 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:45.048608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:45.049537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:45.050345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:45.062529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 140/284] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:45.063476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b9c7-74e4-497f-b0d7-5ba63c09e309 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:45.104739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:45.149118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430ca04-9085-4f3b-bc9c-fb79eed6766b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:45.220588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068a5d-1f6c-43cc-a0e7-d52eb7e5c873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:45.221708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068a5d-1f6c-43cc-a0e7-d52eb7e5c873 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:45.222556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068a5d-1f6c-43cc-a0e7-d52eb7e5c873 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:45.223041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068a5d-1f6c-43cc-a0e7-d52eb7e5c873 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:45.247964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b9c7-74e4-497f-b0d7-5ba63c09e309 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:45.258482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9068a5d-1f6c-43cc-a0e7-d52eb7e5c873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-b9068a5d-1f6c-43cc-a0e7-d52eb7e5c873 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:45.263352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:45.264237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:45.265064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 145/285] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:45.273327 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068a5d-1f6c-43cc-a0e7-d52eb7e5c873 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:45.291300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5bc91-99c9-4882-a306-9c28b751f918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:45.292475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5bc91-99c9-4882-a306-9c28b751f918 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:45.292926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5bc91-99c9-4882-a306-9c28b751f918 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:45.293241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5bc91-99c9-4882-a306-9c28b751f918 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:45.293918 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5bc91-99c9-4882-a306-9c28b751f918 None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:45.294219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5bc91-99c9-4882-a306-9c28b751f918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:45.295650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5bc91-99c9-4882-a306-9c28b751f918 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:45.306474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 141/286] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:45.312409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5bc91-99c9-4882-a306-9c28b751f918 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:45.315286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:45.442927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068a5d-1f6c-43cc-a0e7-d52eb7e5c873 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:45.476075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:45.482134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804711d0-c9a5-482b-9b55-8a0e570024a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:45.482747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804711d0-c9a5-482b-9b55-8a0e570024a1 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:45.483025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804711d0-c9a5-482b-9b55-8a0e570024a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:45.483296 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804711d0-c9a5-482b-9b55-8a0e570024a1 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:45.489158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804711d0-c9a5-482b-9b55-8a0e570024a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:45.489158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804711d0-c9a5-482b-9b55-8a0e570024a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:45.489158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804711d0-c9a5-482b-9b55-8a0e570024a1 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:45.496873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5bc91-99c9-4882-a306-9c28b751f918 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:45.500741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 146/287] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:45.501437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804711d0-c9a5-482b-9b55-8a0e570024a1 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:45.543458 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda766ca-401f-4769-87d7-2f131b412432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:45.544448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda766ca-401f-4769-87d7-2f131b412432 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:45.544932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda766ca-401f-4769-87d7-2f131b412432 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:45.545572 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda766ca-401f-4769-87d7-2f131b412432 None admin] PATH_INFO: `/v3/users/a7af7abd2ae149a79ed5d31ed7d75422` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:45.547083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda766ca-401f-4769-87d7-2f131b412432 None admin] RBAC: Authorizing `identity:get_user(user_id=a7af7abd2ae149a79ed5d31ed7d75422)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:45.548885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda766ca-401f-4769-87d7-2f131b412432 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:45.552073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 142/288] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:45 2022] GET /identity/v3/users/a7af7abd2ae149a79ed5d31ed7d75422 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:45.558949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda766ca-401f-4769-87d7-2f131b412432 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:45.694866 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804711d0-c9a5-482b-9b55-8a0e570024a1 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:45.736933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434c126-299a-4f57-aa61-5bdb97c02ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:45.739246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434c126-299a-4f57-aa61-5bdb97c02ab6 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:45.740772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434c126-299a-4f57-aa61-5bdb97c02ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:45.741297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434c126-299a-4f57-aa61-5bdb97c02ab6 None admin] PATH_INFO: `/v3/users/edc74aeeeadb4d50a50d3b1a5985e4fd` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:45.744242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434c126-299a-4f57-aa61-5bdb97c02ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=edc74aeeeadb4d50a50d3b1a5985e4fd)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:45.745378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:45.746175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434c126-299a-4f57-aa61-5bdb97c02ab6 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:45.754877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 147/289] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:45 2022] GET /identity/v3/users/edc74aeeeadb4d50a50d3b1a5985e4fd => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:45.761280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434c126-299a-4f57-aa61-5bdb97c02ab6 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:45.824481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda766ca-401f-4769-87d7-2f131b412432 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:45.836547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:45.888226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434c126-299a-4f57-aa61-5bdb97c02ab6 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:45.906413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a634b0e-4e39-4f0f-b8a4-d8af961a93d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:45.907385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a634b0e-4e39-4f0f-b8a4-d8af961a93d9 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:45.907889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a634b0e-4e39-4f0f-b8a4-d8af961a93d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:45.908417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a634b0e-4e39-4f0f-b8a4-d8af961a93d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:45.962814 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ec376-b0d8-4b11-a274-e801ab985e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:45.963825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ec376-b0d8-4b11-a274-e801ab985e67 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:45.965979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ec376-b0d8-4b11-a274-e801ab985e67 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:45.966504 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ec376-b0d8-4b11-a274-e801ab985e67 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:45.970620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ec376-b0d8-4b11-a274-e801ab985e67 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:45.972358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ec376-b0d8-4b11-a274-e801ab985e67 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:45.977380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 148/290] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:45 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:45.982638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ec376-b0d8-4b11-a274-e801ab985e67 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:46.047928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a634b0e-4e39-4f0f-b8a4-d8af961a93d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:46.049807 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a634b0e-4e39-4f0f-b8a4-d8af961a93d9 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:46.123914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:46.140123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ec376-b0d8-4b11-a274-e801ab985e67 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:46.182347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:46.189998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2484dae-c7aa-4736-bce0-bd352ec34408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:46.190674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2484dae-c7aa-4736-bce0-bd352ec34408 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:46.190960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2484dae-c7aa-4736-bce0-bd352ec34408 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:46.191238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2484dae-c7aa-4736-bce0-bd352ec34408 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/users/edc74aeeeadb4d50a50d3b1a5985e4fd/roles/1714afd0de144939a3022b0632f03aa0` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:46.196921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2484dae-c7aa-4736-bce0-bd352ec34408 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ef8ff3249dd4e84ad8692c06a441377, user_id=edc74aeeeadb4d50a50d3b1a5985e4fd, role_id=1714afd0de144939a3022b0632f03aa0)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:46.198392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2484dae-c7aa-4736-bce0-bd352ec34408 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:46.205245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 143/291] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:46.211802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a634b0e-4e39-4f0f-b8a4-d8af961a93d9 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:46.310967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 149/292] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:32:45 2022] PUT /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/users/edc74aeeeadb4d50a50d3b1a5985e4fd/roles/1714afd0de144939a3022b0632f03aa0 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:32:46.337083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a634b0e-4e39-4f0f-b8a4-d8af961a93d9 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:46.393674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf5718-2898-4493-8dba-86249ee3bb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:46.394388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf5718-2898-4493-8dba-86249ee3bb6c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:46.394682 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf5718-2898-4493-8dba-86249ee3bb6c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:46.395005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf5718-2898-4493-8dba-86249ee3bb6c None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:46.396179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf5718-2898-4493-8dba-86249ee3bb6c None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:46.398067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf5718-2898-4493-8dba-86249ee3bb6c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:46.400587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 144/293] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:46 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:46.406924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2484dae-c7aa-4736-bce0-bd352ec34408 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:46.536718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2484dae-c7aa-4736-bce0-bd352ec34408 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:46.578888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddf548-6007-45b8-a152-4912a39286c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:46.579500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddf548-6007-45b8-a152-4912a39286c2 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:46.579790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddf548-6007-45b8-a152-4912a39286c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:46.580168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddf548-6007-45b8-a152-4912a39286c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:46.581720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddf548-6007-45b8-a152-4912a39286c2 None admin] RBAC: Adding query filter params (role.id=eccc3f8721b1466b8a1968b77e5c464f, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377, user.id=a7af7abd2ae149a79ed5d31ed7d75422) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:46.582315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddf548-6007-45b8-a152-4912a39286c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:46.585150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddf548-6007-45b8-a152-4912a39286c2 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:46.594425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 150/294] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:46 2022] GET /identity/v3/role_assignments?user.id=a7af7abd2ae149a79ed5d31ed7d75422&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:46.972644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e30b8-4b85-455f-a621-51f80b19fd09 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:46.973206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e30b8-4b85-455f-a621-51f80b19fd09 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:46.973206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e30b8-4b85-455f-a621-51f80b19fd09 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:46.979237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 145/295] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:46.979972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2927326-44c4-4343-b957-1d94d26d0338 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:46.980259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2927326-44c4-4343-b957-1d94d26d0338 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:46.980532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2927326-44c4-4343-b957-1d94d26d0338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:47.000117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a2927326-44c4-4343-b957-1d94d26d0338 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:47.001348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a2927326-44c4-4343-b957-1d94d26d0338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:47.060930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:47.083289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 151/296] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:47.159056 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dc55e-597e-4cf1-a878-cb9cdcd21b9a None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:47.159491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dc55e-597e-4cf1-a878-cb9cdcd21b9a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:47.159865 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dc55e-597e-4cf1-a878-cb9cdcd21b9a None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:47.161690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 146/297] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:47.166640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243e70-3a3f-4629-9d4f-84dbd2527111 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:47.166928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243e70-3a3f-4629-9d4f-84dbd2527111 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:47.167202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243e70-3a3f-4629-9d4f-84dbd2527111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:47.190192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-9b243e70-3a3f-4629-9d4f-84dbd2527111 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:47.191499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9b243e70-3a3f-4629-9d4f-84dbd2527111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:47.275781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 152/298] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:47.318900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a974-c1f3-48fd-ad7e-fe1a1a8a3c13 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:47.319463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a974-c1f3-48fd-ad7e-fe1a1a8a3c13 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:47.319947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a974-c1f3-48fd-ad7e-fe1a1a8a3c13 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:47.322069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 147/299] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:47.327659 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0a241-a273-42e6-878d-0a270781696e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:47.328141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0a241-a273-42e6-878d-0a270781696e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:47.328628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0a241-a273-42e6-878d-0a270781696e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:47.347680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-12e0a241-a273-42e6-878d-0a270781696e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:47.349198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-12e0a241-a273-42e6-878d-0a270781696e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:47.363343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817a974-c1f3-48fd-ad7e-fe1a1a8a3c13 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:47.407462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:47.422990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 153/300] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:47.473126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817a974-c1f3-48fd-ad7e-fe1a1a8a3c13 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:47.514277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f0e07-daf1-4aa5-9b07-9ed4fe01473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:47.514912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f0e07-daf1-4aa5-9b07-9ed4fe01473c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:47.515194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f0e07-daf1-4aa5-9b07-9ed4fe01473c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:47.515470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f0e07-daf1-4aa5-9b07-9ed4fe01473c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:47.526992 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f0e07-daf1-4aa5-9b07-9ed4fe01473c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-dd0f0e07-daf1-4aa5-9b07-9ed4fe01473c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:47.531864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:47.532723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:47.533738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 148/301] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:32:47.538687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0a241-a273-42e6-878d-0a270781696e None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:47.566612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f0e07-daf1-4aa5-9b07-9ed4fe01473c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:47.593439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:47.646486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0a241-a273-42e6-878d-0a270781696e None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:47.689056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1854b-c7a6-46b9-93eb-3386bdd00e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:47.689680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1854b-c7a6-46b9-93eb-3386bdd00e8e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:47.689956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1854b-c7a6-46b9-93eb-3386bdd00e8e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:47.690226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1854b-c7a6-46b9-93eb-3386bdd00e8e None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:47.690879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1854b-c7a6-46b9-93eb-3386bdd00e8e None admin] RBAC: Adding query filter params (name=reader) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:47.691177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1854b-c7a6-46b9-93eb-3386bdd00e8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:47.692560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1854b-c7a6-46b9-93eb-3386bdd00e8e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:47.697995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:47.702120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f0e07-daf1-4aa5-9b07-9ed4fe01473c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:47.709381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 154/302] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:47.711340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1854b-c7a6-46b9-93eb-3386bdd00e8e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:47.744651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca44b3-de78-44b4-810f-2713d5665274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:47.745624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca44b3-de78-44b4-810f-2713d5665274 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:47.746076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca44b3-de78-44b4-810f-2713d5665274 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:47.746543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca44b3-de78-44b4-810f-2713d5665274 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:47.756961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca44b3-de78-44b4-810f-2713d5665274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-81ca44b3-de78-44b4-810f-2713d5665274 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:47.761939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:47.763010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:47.763949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:47.765874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 149/303] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:32:47.766824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca44b3-de78-44b4-810f-2713d5665274 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:47.823377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1854b-c7a6-46b9-93eb-3386bdd00e8e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:47.830215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:47.859904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff05be-7b39-4832-aaf2-eb7c99aabaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:47.860801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff05be-7b39-4832-aaf2-eb7c99aabaf1 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:47.861118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff05be-7b39-4832-aaf2-eb7c99aabaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:47.861647 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff05be-7b39-4832-aaf2-eb7c99aabaf1 None admin] PATH_INFO: `/v3/users/2cb4ed3b6c2843f48fbfae1b58885aa6` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:47.876210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca44b3-de78-44b4-810f-2713d5665274 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:47.877809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff05be-7b39-4832-aaf2-eb7c99aabaf1 None admin] RBAC: Authorizing `identity:get_user(user_id=2cb4ed3b6c2843f48fbfae1b58885aa6)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:47.879393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff05be-7b39-4832-aaf2-eb7c99aabaf1 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:47.885559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 155/304] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:47 2022] GET /identity/v3/users/2cb4ed3b6c2843f48fbfae1b58885aa6 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:47.885559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff05be-7b39-4832-aaf2-eb7c99aabaf1 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:47.915043 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae55a5-454a-44da-97e0-2ee3a90876ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:47.915802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae55a5-454a-44da-97e0-2ee3a90876ff None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:47.916088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae55a5-454a-44da-97e0-2ee3a90876ff None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:47.916368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae55a5-454a-44da-97e0-2ee3a90876ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:47.917525 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae55a5-454a-44da-97e0-2ee3a90876ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:47.919009 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae55a5-454a-44da-97e0-2ee3a90876ff None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:47.920570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 150/305] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:47.921481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae55a5-454a-44da-97e0-2ee3a90876ff None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:47.949754 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:47.994515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff05be-7b39-4832-aaf2-eb7c99aabaf1 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:48.030517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b573bf-af34-432f-a9b4-5e5f1ed3066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:48.031119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b573bf-af34-432f-a9b4-5e5f1ed3066f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:48.031394 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b573bf-af34-432f-a9b4-5e5f1ed3066f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:48.031662 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b573bf-af34-432f-a9b4-5e5f1ed3066f None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:48.032324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b573bf-af34-432f-a9b4-5e5f1ed3066f None admin] RBAC: Adding query filter params (name=member) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:48.032622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b573bf-af34-432f-a9b4-5e5f1ed3066f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:48.033995 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b573bf-af34-432f-a9b4-5e5f1ed3066f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:48.042613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 156/306] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:48.043392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b573bf-af34-432f-a9b4-5e5f1ed3066f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:48.048265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae55a5-454a-44da-97e0-2ee3a90876ff None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:48.071133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:48.089651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f38c4d-aded-4b6c-9e5b-96a6ffbd54e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:48.090777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f38c4d-aded-4b6c-9e5b-96a6ffbd54e8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:48.091251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f38c4d-aded-4b6c-9e5b-96a6ffbd54e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:48.091885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f38c4d-aded-4b6c-9e5b-96a6ffbd54e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:48.093515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f38c4d-aded-4b6c-9e5b-96a6ffbd54e8 None admin] RBAC: Adding query filter params (role.id=1714afd0de144939a3022b0632f03aa0, user.id=2cb4ed3b6c2843f48fbfae1b58885aa6, scope.system=all) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:48.094276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f38c4d-aded-4b6c-9e5b-96a6ffbd54e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:48.096368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f38c4d-aded-4b6c-9e5b-96a6ffbd54e8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:48.103744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 151/307] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 8 18:32:47 2022] GET /identity/v3/role_assignments?user.id=2cb4ed3b6c2843f48fbfae1b58885aa6&scope.system=all&role.id=1714afd0de144939a3022b0632f03aa0 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:48.104755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f38c4d-aded-4b6c-9e5b-96a6ffbd54e8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:48.161886 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b573bf-af34-432f-a9b4-5e5f1ed3066f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:48.210328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d5548-54ef-49be-90d5-1f12f40e45d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:48.211080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d5548-54ef-49be-90d5-1f12f40e45d7 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:48.211369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d5548-54ef-49be-90d5-1f12f40e45d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:48.211646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d5548-54ef-49be-90d5-1f12f40e45d7 None admin] PATH_INFO: `/v3/groups` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:48.212495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d5548-54ef-49be-90d5-1f12f40e45d7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:48.214951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d5548-54ef-49be-90d5-1f12f40e45d7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:48.239204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:48.250437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f38c4d-aded-4b6c-9e5b-96a6ffbd54e8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:48.291039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da7ff0-8496-4cf8-896e-d98a0a69d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:48.291703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da7ff0-8496-4cf8-896e-d98a0a69d062 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:48.291985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da7ff0-8496-4cf8-896e-d98a0a69d062 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:48.292255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da7ff0-8496-4cf8-896e-d98a0a69d062 None admin] PATH_INFO: `/v3/users/620392eb9b154c35ac6dc5bd40c01edc` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:48.297187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 157/308] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Sep 8 18:32:48 2022] POST /identity/v3/groups => generated 229 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:32:48.297375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da7ff0-8496-4cf8-896e-d98a0a69d062 None admin] RBAC: Authorizing `identity:get_user(user_id=620392eb9b154c35ac6dc5bd40c01edc)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:48.297375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da7ff0-8496-4cf8-896e-d98a0a69d062 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:48.297651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 152/309] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:48 2022] GET /identity/v3/users/620392eb9b154c35ac6dc5bd40c01edc => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:48.303525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d5548-54ef-49be-90d5-1f12f40e45d7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:48.408462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:48.425167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d5548-54ef-49be-90d5-1f12f40e45d7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:48.464222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4ba2a-1262-46bd-9675-1398a84a6638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:48.464901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4ba2a-1262-46bd-9675-1398a84a6638 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:48.465212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4ba2a-1262-46bd-9675-1398a84a6638 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:48.465485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4ba2a-1262-46bd-9675-1398a84a6638 None admin] PATH_INFO: `/v3/system/users/620392eb9b154c35ac6dc5bd40c01edc/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:48.467020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4ba2a-1262-46bd-9675-1398a84a6638 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=620392eb9b154c35ac6dc5bd40c01edc, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:48.468671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4ba2a-1262-46bd-9675-1398a84a6638 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:48.543106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 158/310] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Sep 8 18:32:48 2022] PUT /identity/v3/system/users/620392eb9b154c35ac6dc5bd40c01edc/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:32:48.862919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de3666-407c-4da1-9985-56b3a4a5a5bd None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:48.863340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de3666-407c-4da1-9985-56b3a4a5a5bd None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:48.863465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de3666-407c-4da1-9985-56b3a4a5a5bd None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:48.864940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 153/311] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:48.870556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced28b8-cb31-4861-9c03-94e7f9049c6b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:48.870848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced28b8-cb31-4861-9c03-94e7f9049c6b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:48.871122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced28b8-cb31-4861-9c03-94e7f9049c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:48.890880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-0ced28b8-cb31-4861-9c03-94e7f9049c6b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:48.892084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0ced28b8-cb31-4861-9c03-94e7f9049c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:48.964746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 159/312] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:49.232815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de3666-407c-4da1-9985-56b3a4a5a5bd None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:49.340169 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de3666-407c-4da1-9985-56b3a4a5a5bd None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:49.376242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a568d-f1e8-4e2a-969c-5f22b745d7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:49.377274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a568d-f1e8-4e2a-969c-5f22b745d7fb None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:49.377826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a568d-f1e8-4e2a-969c-5f22b745d7fb None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:49.378294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a568d-f1e8-4e2a-969c-5f22b745d7fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:49.389356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a568d-f1e8-4e2a-969c-5f22b745d7fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-603a568d-f1e8-4e2a-969c-5f22b745d7fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:49.394616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:49.395484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:49.396286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:49.404080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 154/313] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:49.405687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced28b8-cb31-4861-9c03-94e7f9049c6b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:49.501537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:49.519666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced28b8-cb31-4861-9c03-94e7f9049c6b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:49.554440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a2b8-1067-4120-8adc-bbb156cb4a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:49.555339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a2b8-1067-4120-8adc-bbb156cb4a92 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:49.555620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a2b8-1067-4120-8adc-bbb156cb4a92 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:49.555891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4a2b8-1067-4120-8adc-bbb156cb4a92 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:49.556555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a2b8-1067-4120-8adc-bbb156cb4a92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:49.556854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a2b8-1067-4120-8adc-bbb156cb4a92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:49.558262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4a2b8-1067-4120-8adc-bbb156cb4a92 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:49.567210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 160/314] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:49.573488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a568d-f1e8-4e2a-969c-5f22b745d7fb None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:49.595319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:49.680417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a568d-f1e8-4e2a-969c-5f22b745d7fb None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:49.715710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389881e-f952-4aa0-8f3f-a35a4316eb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:49.716661 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389881e-f952-4aa0-8f3f-a35a4316eb13 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:49.717210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389881e-f952-4aa0-8f3f-a35a4316eb13 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:49.717918 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389881e-f952-4aa0-8f3f-a35a4316eb13 None admin] PATH_INFO: `/v3/users/edc74aeeeadb4d50a50d3b1a5985e4fd` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:49.719226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389881e-f952-4aa0-8f3f-a35a4316eb13 None admin] RBAC: Authorizing `identity:get_user(user_id=edc74aeeeadb4d50a50d3b1a5985e4fd)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:49.720833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389881e-f952-4aa0-8f3f-a35a4316eb13 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:49.727017 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 155/315] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:49 2022] GET /identity/v3/users/edc74aeeeadb4d50a50d3b1a5985e4fd => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:49.729306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a2b8-1067-4120-8adc-bbb156cb4a92 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:49.830732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4a2b8-1067-4120-8adc-bbb156cb4a92 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:49.865437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c475ad-1567-4177-99d4-a6654c071a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:49.866049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c475ad-1567-4177-99d4-a6654c071a91 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:49.866328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c475ad-1567-4177-99d4-a6654c071a91 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:49.866599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c475ad-1567-4177-99d4-a6654c071a91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:49.885413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:49.938613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c475ad-1567-4177-99d4-a6654c071a91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:49.940068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c475ad-1567-4177-99d4-a6654c071a91 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:50.041508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 161/316] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:50.047315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389881e-f952-4aa0-8f3f-a35a4316eb13 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:50.140876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:50.156904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389881e-f952-4aa0-8f3f-a35a4316eb13 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:50.193111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5936e-94f6-437c-af89-4a82ea6380c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:50.194048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5936e-94f6-437c-af89-4a82ea6380c4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:50.194540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5936e-94f6-437c-af89-4a82ea6380c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:50.194991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5936e-94f6-437c-af89-4a82ea6380c4 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:50.196864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5936e-94f6-437c-af89-4a82ea6380c4 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:50.198512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5936e-94f6-437c-af89-4a82ea6380c4 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:50.205257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 156/317] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:50 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:50.207018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c475ad-1567-4177-99d4-a6654c071a91 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:50.262341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:50.319725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c475ad-1567-4177-99d4-a6654c071a91 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:50.357963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0d775-4424-4ff5-8e9e-dc1c5d01f6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:50.358654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0d775-4424-4ff5-8e9e-dc1c5d01f6d6 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:50.358936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0d775-4424-4ff5-8e9e-dc1c5d01f6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:50.359208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0d775-4424-4ff5-8e9e-dc1c5d01f6d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:50.359967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0d775-4424-4ff5-8e9e-dc1c5d01f6d6 None admin] RBAC: Adding query filter params (role.id=1714afd0de144939a3022b0632f03aa0, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377, user.id=edc74aeeeadb4d50a50d3b1a5985e4fd) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:50.360278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0d775-4424-4ff5-8e9e-dc1c5d01f6d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:50.361806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0d775-4424-4ff5-8e9e-dc1c5d01f6d6 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:50.369102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 162/318] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:32:50 2022] GET /identity/v3/role_assignments?user.id=edc74aeeeadb4d50a50d3b1a5985e4fd&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=1714afd0de144939a3022b0632f03aa0 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:50.477751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64976971-93a5-4426-b973-a0e0bbfed425 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:50.478237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64976971-93a5-4426-b973-a0e0bbfed425 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:50.478237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64976971-93a5-4426-b973-a0e0bbfed425 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:50.479875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 157/319] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:50.485466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b5bab-76f2-42c8-bf03-278b4849e860 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:50.485798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b5bab-76f2-42c8-bf03-278b4849e860 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:50.486017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b5bab-76f2-42c8-bf03-278b4849e860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:50.506026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-112b5bab-76f2-42c8-bf03-278b4849e860 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:50.507268 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-112b5bab-76f2-42c8-bf03-278b4849e860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:50.584623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 163/320] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:50.651729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fa0fa-7526-470c-a402-ceee2796b3d9 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:50.652049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fa0fa-7526-470c-a402-ceee2796b3d9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:50.652294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fa0fa-7526-470c-a402-ceee2796b3d9 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:50.653854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 158/321] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:50.659022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f51d5-87d3-44f4-855a-5bdcec7d7de8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:50.659312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f51d5-87d3-44f4-855a-5bdcec7d7de8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:50.659591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f51d5-87d3-44f4-855a-5bdcec7d7de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:50.678223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b55f51d5-87d3-44f4-855a-5bdcec7d7de8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:50.679408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b55f51d5-87d3-44f4-855a-5bdcec7d7de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:50.709602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:50.757315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 164/322] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:50.846660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133a452-1848-4978-9dd3-6f9ba596c383 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:50.846963 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133a452-1848-4978-9dd3-6f9ba596c383 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:50.847233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133a452-1848-4978-9dd3-6f9ba596c383 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:50.848742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 159/323] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:50.854079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133a452-1848-4978-9dd3-6f9ba596c383 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:50.854341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee54a21-9d74-44d3-a457-eea073403342 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:50.854631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee54a21-9d74-44d3-a457-eea073403342 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:50.854912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee54a21-9d74-44d3-a457-eea073403342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:50.873833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1ee54a21-9d74-44d3-a457-eea073403342 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:50.875033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1ee54a21-9d74-44d3-a457-eea073403342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:50.946831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 165/324] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:50.977579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133a452-1848-4978-9dd3-6f9ba596c383 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:51.013845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559be53-a101-4aad-86df-8fffed8c6273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:51.014456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559be53-a101-4aad-86df-8fffed8c6273 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:51.014738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559be53-a101-4aad-86df-8fffed8c6273 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:51.015059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559be53-a101-4aad-86df-8fffed8c6273 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:51.024672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee54a21-9d74-44d3-a457-eea073403342 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:51.026442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559be53-a101-4aad-86df-8fffed8c6273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-d559be53-a101-4aad-86df-8fffed8c6273 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:51.031302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:51.032164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:51.033167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 160/325] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:32:51.037889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559be53-a101-4aad-86df-8fffed8c6273 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:51.054529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:51.140716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559be53-a101-4aad-86df-8fffed8c6273 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:51.144349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:51.151926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee54a21-9d74-44d3-a457-eea073403342 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:51.177594 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff18e1-3b6c-449c-8434-44a8bc7ddd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:51.178194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff18e1-3b6c-449c-8434-44a8bc7ddd9d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:51.178469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff18e1-3b6c-449c-8434-44a8bc7ddd9d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:51.178738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff18e1-3b6c-449c-8434-44a8bc7ddd9d None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:51.179391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff18e1-3b6c-449c-8434-44a8bc7ddd9d None admin] RBAC: Adding query filter params (name=reader) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:51.179928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff18e1-3b6c-449c-8434-44a8bc7ddd9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:51.181355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff18e1-3b6c-449c-8434-44a8bc7ddd9d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:51.188521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e88813-6214-4b64-b0e3-133f84d8d206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:51.189190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e88813-6214-4b64-b0e3-133f84d8d206 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:51.189732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e88813-6214-4b64-b0e3-133f84d8d206 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:51.189836 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 161/326] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:51.189930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e88813-6214-4b64-b0e3-133f84d8d206 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:51.190888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e88813-6214-4b64-b0e3-133f84d8d206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:51.192323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e88813-6214-4b64-b0e3-133f84d8d206 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:51.194046 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 166/327] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:32:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:51.195035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff18e1-3b6c-449c-8434-44a8bc7ddd9d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:51.199780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e88813-6214-4b64-b0e3-133f84d8d206 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:51.272126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:51.295299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff18e1-3b6c-449c-8434-44a8bc7ddd9d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:51.302156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e88813-6214-4b64-b0e3-133f84d8d206 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:51.328705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b84474-9c08-4730-804e-43bb6175a1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:51.329428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b84474-9c08-4730-804e-43bb6175a1d8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:51.329781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b84474-9c08-4730-804e-43bb6175a1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:51.330119 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b84474-9c08-4730-804e-43bb6175a1d8 None admin] PATH_INFO: `/v3/users/2cb4ed3b6c2843f48fbfae1b58885aa6` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:51.331227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b84474-9c08-4730-804e-43bb6175a1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=2cb4ed3b6c2843f48fbfae1b58885aa6)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:51.332650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b84474-9c08-4730-804e-43bb6175a1d8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:51.334245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 162/328] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:51 2022] GET /identity/v3/users/2cb4ed3b6c2843f48fbfae1b58885aa6 => generated 293 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:51.335292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad4cde-e0d8-4046-9be8-67c7d88fba79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:51.336436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad4cde-e0d8-4046-9be8-67c7d88fba79 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:51.336758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad4cde-e0d8-4046-9be8-67c7d88fba79 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:51.337185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad4cde-e0d8-4046-9be8-67c7d88fba79 None admin] PATH_INFO: `/v3/groups` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:51.337821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad4cde-e0d8-4046-9be8-67c7d88fba79 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:51.339251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad4cde-e0d8-4046-9be8-67c7d88fba79 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:51.340315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b84474-9c08-4730-804e-43bb6175a1d8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:51.431267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 167/329] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Sep 8 18:32:51 2022] POST /identity/v3/groups => generated 226 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:32:51.431511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad4cde-e0d8-4046-9be8-67c7d88fba79 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:51.446842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b84474-9c08-4730-804e-43bb6175a1d8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:51.476256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:51.488165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c608ad-c3a0-4ff8-a505-392b84ac0ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:51.489768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c608ad-c3a0-4ff8-a505-392b84ac0ea2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:51.489768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c608ad-c3a0-4ff8-a505-392b84ac0ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:51.489768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c608ad-c3a0-4ff8-a505-392b84ac0ea2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:51.503033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c608ad-c3a0-4ff8-a505-392b84ac0ea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-63c608ad-c3a0-4ff8-a505-392b84ac0ea2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:51.509155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:51.510313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:51.511122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 163/330] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:51.516413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c608ad-c3a0-4ff8-a505-392b84ac0ea2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:51.520500 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:51.574392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad4cde-e0d8-4046-9be8-67c7d88fba79 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:51.616894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c270e-2e24-4432-9068-9b0d0cfebf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:51.618410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c270e-2e24-4432-9068-9b0d0cfebf6c None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:51.618698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c270e-2e24-4432-9068-9b0d0cfebf6c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:51.618982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c270e-2e24-4432-9068-9b0d0cfebf6c None admin] PATH_INFO: `/v3/system/users/2cb4ed3b6c2843f48fbfae1b58885aa6/roles/1714afd0de144939a3022b0632f03aa0` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:51.620426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c270e-2e24-4432-9068-9b0d0cfebf6c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2cb4ed3b6c2843f48fbfae1b58885aa6, role_id=1714afd0de144939a3022b0632f03aa0)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:51.621845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c270e-2e24-4432-9068-9b0d0cfebf6c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:51.642093 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c608ad-c3a0-4ff8-a505-392b84ac0ea2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:51.677672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cebce-74e2-4737-a169-a370dfb6d18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:51.678288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cebce-74e2-4737-a169-a370dfb6d18c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:51.678571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cebce-74e2-4737-a169-a370dfb6d18c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:51.678864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cebce-74e2-4737-a169-a370dfb6d18c None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:51.679601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cebce-74e2-4737-a169-a370dfb6d18c None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:51.680486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cebce-74e2-4737-a169-a370dfb6d18c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:51.681935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cebce-74e2-4737-a169-a370dfb6d18c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:51.690606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 164/331] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:51.692978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 168/332] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Sep 8 18:32:51 2022] PUT /identity/v3/system/users/2cb4ed3b6c2843f48fbfae1b58885aa6/roles/1714afd0de144939a3022b0632f03aa0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:32:51.701515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cebce-74e2-4737-a169-a370dfb6d18c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:51.779849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:51.822326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cebce-74e2-4737-a169-a370dfb6d18c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:51.866390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e3939-025d-4f37-ad64-cdca72660847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:51.869342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3939-025d-4f37-ad64-cdca72660847 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:51.869342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3939-025d-4f37-ad64-cdca72660847 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:51.869342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e3939-025d-4f37-ad64-cdca72660847 None admin] PATH_INFO: `/v3/users/620392eb9b154c35ac6dc5bd40c01edc` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:51.871758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e3939-025d-4f37-ad64-cdca72660847 None admin] RBAC: Authorizing `identity:get_user(user_id=620392eb9b154c35ac6dc5bd40c01edc)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:51.871758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e3939-025d-4f37-ad64-cdca72660847 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:51.874293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 165/333] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:51 2022] GET /identity/v3/users/620392eb9b154c35ac6dc5bd40c01edc => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:51.880700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c270e-2e24-4432-9068-9b0d0cfebf6c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:51.984636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:51.999198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c270e-2e24-4432-9068-9b0d0cfebf6c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:52.038493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396d46d-5548-4291-b9ce-f1f62b29c37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:52.039672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396d46d-5548-4291-b9ce-f1f62b29c37c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:52.039950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396d46d-5548-4291-b9ce-f1f62b29c37c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:52.040227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396d46d-5548-4291-b9ce-f1f62b29c37c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:52.040953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396d46d-5548-4291-b9ce-f1f62b29c37c None admin] RBAC: Adding query filter params (role.id=eccc3f8721b1466b8a1968b77e5c464f, user.id=620392eb9b154c35ac6dc5bd40c01edc, scope.system=all) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:52.041289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396d46d-5548-4291-b9ce-f1f62b29c37c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:52.042794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396d46d-5548-4291-b9ce-f1f62b29c37c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:52.055890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 169/334] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 8 18:32:51 2022] GET /identity/v3/role_assignments?user.id=620392eb9b154c35ac6dc5bd40c01edc&scope.system=all&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:54.054008 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f4e8-7aa4-4e22-afc2-eeb95a280015 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:54.054598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f4e8-7aa4-4e22-afc2-eeb95a280015 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:54.054598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f4e8-7aa4-4e22-afc2-eeb95a280015 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:54.056295 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 166/335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:54.061550 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c825-cd2e-413f-9b9f-effb5b409a7c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:54.061973 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c825-cd2e-413f-9b9f-effb5b409a7c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:54.062157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c825-cd2e-413f-9b9f-effb5b409a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:54.085601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7c33c825-cd2e-413f-9b9f-effb5b409a7c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:54.086878 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7c33c825-cd2e-413f-9b9f-effb5b409a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:54.185076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 170/336] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:54.367113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8cf88-e03c-410a-b752-8b3a3be0cff9 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:54.367441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8cf88-e03c-410a-b752-8b3a3be0cff9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:54.367615 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8cf88-e03c-410a-b752-8b3a3be0cff9 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:54.369082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 167/337] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:54.374787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661924e1-5d71-4b00-b4cf-20115f588633 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:54.375081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661924e1-5d71-4b00-b4cf-20115f588633 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:54.375361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661924e1-5d71-4b00-b4cf-20115f588633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:54.399646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-661924e1-5d71-4b00-b4cf-20115f588633 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:54.400901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-661924e1-5d71-4b00-b4cf-20115f588633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:54.432312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6936ea-9748-4201-bcc6-3a3d1274e660 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:54.432918 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6936ea-9748-4201-bcc6-3a3d1274e660 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:54.433404 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6936ea-9748-4201-bcc6-3a3d1274e660 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:54.435170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 168/338] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:54.440013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c8f17-a47f-45cc-a159-a9a047dbba4a None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:54.440514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c8f17-a47f-45cc-a159-a9a047dbba4a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:54.440978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c8f17-a47f-45cc-a159-a9a047dbba4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:54.463931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-6a6c8f17-a47f-45cc-a159-a9a047dbba4a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:54.469014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6a6c8f17-a47f-45cc-a159-a9a047dbba4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:54.506919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 171/339] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:54.556403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 169/340] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:54.605773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661924e1-5d71-4b00-b4cf-20115f588633 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:54.614956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:54.735174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661924e1-5d71-4b00-b4cf-20115f588633 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:54.774252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df6af9-aa0b-4a39-9a4c-7c6b0b842eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:54.774897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df6af9-aa0b-4a39-9a4c-7c6b0b842eac None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:54.775173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df6af9-aa0b-4a39-9a4c-7c6b0b842eac None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:54.775440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df6af9-aa0b-4a39-9a4c-7c6b0b842eac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:54.786200 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df6af9-aa0b-4a39-9a4c-7c6b0b842eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-02df6af9-aa0b-4a39-9a4c-7c6b0b842eac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:54.791158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:54.792037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:54.792874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 172/341] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:54.798203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c8f17-a47f-45cc-a159-a9a047dbba4a None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:54.834831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673b202-bdc7-45a0-8f25-d7eba6df64a2 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:54.835153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673b202-bdc7-45a0-8f25-d7eba6df64a2 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:54.835369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673b202-bdc7-45a0-8f25-d7eba6df64a2 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:54.837528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 173/342] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:54.843431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c6ccf-df15-4533-a248-766d86102c43 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:54.843716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c6ccf-df15-4533-a248-766d86102c43 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:54.843991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c6ccf-df15-4533-a248-766d86102c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:54.871613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-084c6ccf-df15-4533-a248-766d86102c43 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:54.872826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-084c6ccf-df15-4533-a248-766d86102c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:54.893990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:54.966328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c8f17-a47f-45cc-a159-a9a047dbba4a None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:55.010180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:55.022402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 174/343] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:55.023311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c6ccf-df15-4533-a248-766d86102c43 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:55.048463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b7fb-b429-4e1d-963e-ea5441700a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:55.049175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b7fb-b429-4e1d-963e-ea5441700a81 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:55.049462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b7fb-b429-4e1d-963e-ea5441700a81 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:55.049758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b7fb-b429-4e1d-963e-ea5441700a81 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:55.052238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066b7fb-b429-4e1d-963e-ea5441700a81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:55.052550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066b7fb-b429-4e1d-963e-ea5441700a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:55.056230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066b7fb-b429-4e1d-963e-ea5441700a81 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:55.073218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 170/344] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:55.077471 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b7fb-b429-4e1d-963e-ea5441700a81 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:55.244006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c6ccf-df15-4533-a248-766d86102c43 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:55.301701 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7466d-4b77-49f1-8329-3c86c15d51a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:55.302838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7466d-4b77-49f1-8329-3c86c15d51a8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:55.303343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7466d-4b77-49f1-8329-3c86c15d51a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:55.306100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7466d-4b77-49f1-8329-3c86c15d51a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:55.311980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b7fb-b429-4e1d-963e-ea5441700a81 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:55.330183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7466d-4b77-49f1-8329-3c86c15d51a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-0ce7466d-4b77-49f1-8329-3c86c15d51a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:55.339765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:55.342044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 175/345] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:32:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:55.344300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7466d-4b77-49f1-8329-3c86c15d51a8 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:55.367624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:55.383374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7ce13-e2e5-48f4-a9c5-27ca3eeee35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:55.384369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7ce13-e2e5-48f4-a9c5-27ca3eeee35f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:55.384878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7ce13-e2e5-48f4-a9c5-27ca3eeee35f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:55.385480 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7ce13-e2e5-48f4-a9c5-27ca3eeee35f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:55.398078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7ce13-e2e5-48f4-a9c5-27ca3eeee35f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-bdf7ce13-e2e5-48f4-a9c5-27ca3eeee35f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:55.405512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:55.406462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:55.407353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:32:55.411120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 171/346] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:55.412157 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7ce13-e2e5-48f4-a9c5-27ca3eeee35f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:55.437790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:55.487249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7466d-4b77-49f1-8329-3c86c15d51a8 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:55.534432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcd852-e778-42c8-a198-54bec45ff575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:55.535079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcd852-e778-42c8-a198-54bec45ff575 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:55.535414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcd852-e778-42c8-a198-54bec45ff575 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:55.535759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcd852-e778-42c8-a198-54bec45ff575 None admin] PATH_INFO: `/v3/users/2cb4ed3b6c2843f48fbfae1b58885aa6` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:55.537086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcd852-e778-42c8-a198-54bec45ff575 None admin] RBAC: Authorizing `identity:get_user(user_id=2cb4ed3b6c2843f48fbfae1b58885aa6)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:55.538681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcd852-e778-42c8-a198-54bec45ff575 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:55.540940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 176/347] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:32:55 2022] GET /identity/v3/users/2cb4ed3b6c2843f48fbfae1b58885aa6 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:55.541879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcd852-e778-42c8-a198-54bec45ff575 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:55.553432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7ce13-e2e5-48f4-a9c5-27ca3eeee35f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:55.614647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33dfd2-ddd0-46a1-8eb3-006e86e86650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:55.615282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33dfd2-ddd0-46a1-8eb3-006e86e86650 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:55.615563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33dfd2-ddd0-46a1-8eb3-006e86e86650 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:55.615842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33dfd2-ddd0-46a1-8eb3-006e86e86650 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:55.616524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33dfd2-ddd0-46a1-8eb3-006e86e86650 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:55.616827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33dfd2-ddd0-46a1-8eb3-006e86e86650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:55.619417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33dfd2-ddd0-46a1-8eb3-006e86e86650 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:55.631279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 172/348] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:32:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:55.634017 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33dfd2-ddd0-46a1-8eb3-006e86e86650 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:55.688017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcd852-e778-42c8-a198-54bec45ff575 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:55.706723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:55.723892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:55.741130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200f5e-c0be-4bc5-879d-16c338a6aa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:55.748739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200f5e-c0be-4bc5-879d-16c338a6aa71 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:55.749073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200f5e-c0be-4bc5-879d-16c338a6aa71 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:55.749323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200f5e-c0be-4bc5-879d-16c338a6aa71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:55.775189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72200f5e-c0be-4bc5-879d-16c338a6aa71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-72200f5e-c0be-4bc5-879d-16c338a6aa71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:55.783331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:55.784242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:55.785016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 177/349] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:55.785699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200f5e-c0be-4bc5-879d-16c338a6aa71 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:55.800655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33dfd2-ddd0-46a1-8eb3-006e86e86650 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:55.844190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1498cf-c1b2-4ded-b7cb-f87202ab3553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:55.845202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1498cf-c1b2-4ded-b7cb-f87202ab3553 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:55.845679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1498cf-c1b2-4ded-b7cb-f87202ab3553 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:55.846320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1498cf-c1b2-4ded-b7cb-f87202ab3553 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:55.847225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1498cf-c1b2-4ded-b7cb-f87202ab3553 None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:55.847720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1498cf-c1b2-4ded-b7cb-f87202ab3553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:55.849413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1498cf-c1b2-4ded-b7cb-f87202ab3553 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:55.863270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 173/350] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:55.864434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1498cf-c1b2-4ded-b7cb-f87202ab3553 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:55.919329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200f5e-c0be-4bc5-879d-16c338a6aa71 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:55.965606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9fd29-5076-4c8c-a664-dd56245a6c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:55.966226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9fd29-5076-4c8c-a664-dd56245a6c8d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:55.966509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9fd29-5076-4c8c-a664-dd56245a6c8d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:55.966782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9fd29-5076-4c8c-a664-dd56245a6c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:55.967771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9fd29-5076-4c8c-a664-dd56245a6c8d None admin] RBAC: Adding query filter params (role.id=1714afd0de144939a3022b0632f03aa0, user.id=2cb4ed3b6c2843f48fbfae1b58885aa6, scope.system=all) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:55.968084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9fd29-5076-4c8c-a664-dd56245a6c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:55.969564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9fd29-5076-4c8c-a664-dd56245a6c8d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:55.988315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 178/351] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 8 18:32:55 2022] GET /identity/v3/role_assignments?user.id=2cb4ed3b6c2843f48fbfae1b58885aa6&scope.system=all&role.id=1714afd0de144939a3022b0632f03aa0 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:55.989289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9fd29-5076-4c8c-a664-dd56245a6c8d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:56.008477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1498cf-c1b2-4ded-b7cb-f87202ab3553 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:56.026845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:56.050308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:56.079684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98addf-38d6-4e1c-b19d-e7ad36bad872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:56.080408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98addf-38d6-4e1c-b19d-e7ad36bad872 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:56.080710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98addf-38d6-4e1c-b19d-e7ad36bad872 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:56.081608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98addf-38d6-4e1c-b19d-e7ad36bad872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:56.124274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9fd29-5076-4c8c-a664-dd56245a6c8d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:56.162858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159a4e4-b729-4794-8478-79f962f882a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:56.164144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159a4e4-b729-4794-8478-79f962f882a8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:56.164736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159a4e4-b729-4794-8478-79f962f882a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:56.165313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159a4e4-b729-4794-8478-79f962f882a8 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:56.166225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159a4e4-b729-4794-8478-79f962f882a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:56.166739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159a4e4-b729-4794-8478-79f962f882a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:56.168473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159a4e4-b729-4794-8478-79f962f882a8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:56.178602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 179/352] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:32:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:56.180167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159a4e4-b729-4794-8478-79f962f882a8 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:56.215294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98addf-38d6-4e1c-b19d-e7ad36bad872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:56.219569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98addf-38d6-4e1c-b19d-e7ad36bad872 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:56.325466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159a4e4-b729-4794-8478-79f962f882a8 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:56.328659 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:56.365888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 174/353] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:56.367388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98addf-38d6-4e1c-b19d-e7ad36bad872 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:56.372852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afcd0c-27c1-4c0c-bd27-fb577d15f117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:56.373520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afcd0c-27c1-4c0c-bd27-fb577d15f117 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:56.373790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afcd0c-27c1-4c0c-bd27-fb577d15f117 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:56.374067 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afcd0c-27c1-4c0c-bd27-fb577d15f117 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:56.381812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:56.458369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afcd0c-27c1-4c0c-bd27-fb577d15f117 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:56.459826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afcd0c-27c1-4c0c-bd27-fb577d15f117 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:56.496879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98addf-38d6-4e1c-b19d-e7ad36bad872 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:56.540351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9882ad0-4221-4bd5-8211-a180a2fff3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:56.540956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9882ad0-4221-4bd5-8211-a180a2fff3ce None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:56.541301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9882ad0-4221-4bd5-8211-a180a2fff3ce None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:56.541571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9882ad0-4221-4bd5-8211-a180a2fff3ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:56.596794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 180/354] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:56.597431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e36857-dc5f-468e-875c-88a342dee18d None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:56.597431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e36857-dc5f-468e-875c-88a342dee18d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:56.597740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e36857-dc5f-468e-875c-88a342dee18d None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:56.599281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 181/355] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:56.599676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e36857-dc5f-468e-875c-88a342dee18d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:56.604789 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:56.647156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9882ad0-4221-4bd5-8211-a180a2fff3ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:56.649238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9882ad0-4221-4bd5-8211-a180a2fff3ce None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:56.662818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:56.732183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e36857-dc5f-468e-875c-88a342dee18d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:56.773921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227d2ad-c101-4363-be9e-cbc66b553b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:56.774910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d2ad-c101-4363-be9e-cbc66b553b27 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:56.775293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d2ad-c101-4363-be9e-cbc66b553b27 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:56.775570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227d2ad-c101-4363-be9e-cbc66b553b27 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:56.776797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227d2ad-c101-4363-be9e-cbc66b553b27 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:56.778296 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227d2ad-c101-4363-be9e-cbc66b553b27 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:56.779630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 175/356] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:56.780200 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 182/357] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:56 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:56.780912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9882ad0-4221-4bd5-8211-a180a2fff3ce None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:56.781510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e241-bec5-4242-b4ff-edacd5c41ad9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:56.781832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e241-bec5-4242-b4ff-edacd5c41ad9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:56.782051 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e241-bec5-4242-b4ff-edacd5c41ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:56.801190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8d85e241-bec5-4242-b4ff-edacd5c41ad9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:56.801655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8d85e241-bec5-4242-b4ff-edacd5c41ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:56.889087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 183/358] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:56.889087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e241-bec5-4242-b4ff-edacd5c41ad9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:56.889087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:56.904448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9882ad0-4221-4bd5-8211-a180a2fff3ce None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:56.918143 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:56.950134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f248d-9cb7-40ef-9327-08ed0a3da4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:56.951076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f248d-9cb7-40ef-9327-08ed0a3da4f0 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:56.951633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f248d-9cb7-40ef-9327-08ed0a3da4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:56.952111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f248d-9cb7-40ef-9327-08ed0a3da4f0 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:56.953625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f248d-9cb7-40ef-9327-08ed0a3da4f0 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:56.954795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f248d-9cb7-40ef-9327-08ed0a3da4f0 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:56.956633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 176/359] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:56 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:56.957229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f248d-9cb7-40ef-9327-08ed0a3da4f0 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:57.009944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e241-bec5-4242-b4ff-edacd5c41ad9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:57.058046 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a390e59-a8b8-4472-ba27-10fe9073339d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:57.058046 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a390e59-a8b8-4472-ba27-10fe9073339d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:57.058439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a390e59-a8b8-4472-ba27-10fe9073339d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:57.058634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a390e59-a8b8-4472-ba27-10fe9073339d None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:57.060856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a390e59-a8b8-4472-ba27-10fe9073339d None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:57.062738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a390e59-a8b8-4472-ba27-10fe9073339d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:57.064836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 184/360] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:56 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:57.066346 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4070e7-0967-40cc-a493-5cea94d5c65c None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:57.066628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4070e7-0967-40cc-a493-5cea94d5c65c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:57.066903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4070e7-0967-40cc-a493-5cea94d5c65c None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:57.068806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 185/361] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:32:57.074315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4070e7-0967-40cc-a493-5cea94d5c65c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:57.093520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f248d-9cb7-40ef-9327-08ed0a3da4f0 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:57.136402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0308646-ddb0-45eb-855b-7ac8316cb276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:57.137037 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0308646-ddb0-45eb-855b-7ac8316cb276 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:57.137355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0308646-ddb0-45eb-855b-7ac8316cb276 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:57.137632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0308646-ddb0-45eb-855b-7ac8316cb276 None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:57.145693 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0308646-ddb0-45eb-855b-7ac8316cb276 None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:57.148538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0308646-ddb0-45eb-855b-7ac8316cb276 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:57.150676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 177/362] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:32:56 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:57.151460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0308646-ddb0-45eb-855b-7ac8316cb276 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:57.164752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:57.203861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:57.206328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4070e7-0967-40cc-a493-5cea94d5c65c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:57.267130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e593797-e9c2-4f7f-a759-1fe7b5188ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:57.267130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593797-e9c2-4f7f-a759-1fe7b5188ad8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:57.267130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593797-e9c2-4f7f-a759-1fe7b5188ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:57.267130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593797-e9c2-4f7f-a759-1fe7b5188ad8 None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:57.268815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e593797-e9c2-4f7f-a759-1fe7b5188ad8 None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:57.269610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e593797-e9c2-4f7f-a759-1fe7b5188ad8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:57.271618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 186/363] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:32:57 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:57.272292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5014fdc-9657-47d4-96d4-9a4e3c6de4fc None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:57.272571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5014fdc-9657-47d4-96d4-9a4e3c6de4fc None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:57.272845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5014fdc-9657-47d4-96d4-9a4e3c6de4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:57.285972 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0308646-ddb0-45eb-855b-7ac8316cb276 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:57.300535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d5014fdc-9657-47d4-96d4-9a4e3c6de4fc None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:32:57.302134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d5014fdc-9657-47d4-96d4-9a4e3c6de4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:32:57.348279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598b768-ec3f-4493-8ec5-cb01fd339b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:57.348899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598b768-ec3f-4493-8ec5-cb01fd339b81 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:57.349217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598b768-ec3f-4493-8ec5-cb01fd339b81 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:57.349494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598b768-ec3f-4493-8ec5-cb01fd339b81 None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:57.352105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598b768-ec3f-4493-8ec5-cb01fd339b81 None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:57.353742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598b768-ec3f-4493-8ec5-cb01fd339b81 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:57.356527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 178/364] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:32:57 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:57.359223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598b768-ec3f-4493-8ec5-cb01fd339b81 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:57.406362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 187/365] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:32:57.406362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5014fdc-9657-47d4-96d4-9a4e3c6de4fc None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:57.503013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598b768-ec3f-4493-8ec5-cb01fd339b81 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:57.505009 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:57.517361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:57.558697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5014fdc-9657-47d4-96d4-9a4e3c6de4fc None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:57.561544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4bfbe-a88c-4a57-8c81-e36f184a0abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:57.562650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4bfbe-a88c-4a57-8c81-e36f184a0abc None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:57.562650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4bfbe-a88c-4a57-8c81-e36f184a0abc None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:57.562650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4bfbe-a88c-4a57-8c81-e36f184a0abc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:57.563404 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4bfbe-a88c-4a57-8c81-e36f184a0abc None admin] RBAC: Adding query filter params (group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role.id=6b3a5d54d6044e9b8d8270e7ff3145e8, scope.project.id=c59969df6296450f916add4011ddf885) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:57.563707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4bfbe-a88c-4a57-8c81-e36f184a0abc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:57.566051 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4bfbe-a88c-4a57-8c81-e36f184a0abc None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:57.580217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 179/366] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 18:32:57 2022] GET /identity/v3/role_assignments?group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d&scope.project.id=c59969df6296450f916add4011ddf885&role.id=6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:57.581877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4bfbe-a88c-4a57-8c81-e36f184a0abc None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:57.625022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b67c28-d92a-4bac-ad94-d2e2b3d9d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:57.626107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67c28-d92a-4bac-ad94-d2e2b3d9d57e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:57.626586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67c28-d92a-4bac-ad94-d2e2b3d9d57e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:57.627083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67c28-d92a-4bac-ad94-d2e2b3d9d57e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:57.645670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b67c28-d92a-4bac-ad94-d2e2b3d9d57e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-30b67c28-d92a-4bac-ad94-d2e2b3d9d57e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:57.655071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:57.656020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:32:57.656802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:57.659331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 188/367] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:32:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:57.660332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b67c28-d92a-4bac-ad94-d2e2b3d9d57e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:57.750719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4bfbe-a88c-4a57-8c81-e36f184a0abc None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:57.795579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e486a-40d8-400b-9556-f75950f195bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:57.796268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e486a-40d8-400b-9556-f75950f195bc None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:57.796551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e486a-40d8-400b-9556-f75950f195bc None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:57.796827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e486a-40d8-400b-9556-f75950f195bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:57.797636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e486a-40d8-400b-9556-f75950f195bc None admin] RBAC: Adding query filter params (group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role.id=eccc3f8721b1466b8a1968b77e5c464f, scope.project.id=c59969df6296450f916add4011ddf885) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:57.797949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e486a-40d8-400b-9556-f75950f195bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:57.799416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e486a-40d8-400b-9556-f75950f195bc None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:57.806755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 180/368] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 18:32:57 2022] GET /identity/v3/role_assignments?group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d&scope.project.id=c59969df6296450f916add4011ddf885&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:57.807608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e486a-40d8-400b-9556-f75950f195bc None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:57.823983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b67c28-d92a-4bac-ad94-d2e2b3d9d57e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:57.862636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:57.870266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f77fbd-6e9c-4192-b2e1-f65b839db63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:57.870894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f77fbd-6e9c-4192-b2e1-f65b839db63d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:57.871175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f77fbd-6e9c-4192-b2e1-f65b839db63d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:57.871449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f77fbd-6e9c-4192-b2e1-f65b839db63d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:57.872182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f77fbd-6e9c-4192-b2e1-f65b839db63d None admin] RBAC: Adding query filter params (group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role.id=eccc3f8721b1466b8a1968b77e5c464f, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:57.872564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f77fbd-6e9c-4192-b2e1-f65b839db63d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:57.873976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f77fbd-6e9c-4192-b2e1-f65b839db63d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:57.880300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 189/369] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 18:32:57 2022] GET /identity/v3/role_assignments?group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:57.881076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f77fbd-6e9c-4192-b2e1-f65b839db63d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:57.888515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:58.015146 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e486a-40d8-400b-9556-f75950f195bc None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:58.061530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30927fb1-845f-4145-aff9-9c8adc04cb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:58.062996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927fb1-845f-4145-aff9-9c8adc04cb72 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:58.063277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927fb1-845f-4145-aff9-9c8adc04cb72 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:58.063552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927fb1-845f-4145-aff9-9c8adc04cb72 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:58.064243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30927fb1-845f-4145-aff9-9c8adc04cb72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:58.064557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30927fb1-845f-4145-aff9-9c8adc04cb72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:58.066798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30927fb1-845f-4145-aff9-9c8adc04cb72 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:58.086171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 181/370] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:32:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:58.092812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30927fb1-845f-4145-aff9-9c8adc04cb72 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:58.138893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f77fbd-6e9c-4192-b2e1-f65b839db63d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:58.199193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5b9ef-cd39-4915-8b72-73209d7b6089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:58.200299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5b9ef-cd39-4915-8b72-73209d7b6089 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:58.200898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5b9ef-cd39-4915-8b72-73209d7b6089 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:58.201290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5b9ef-cd39-4915-8b72-73209d7b6089 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:58.222044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5b9ef-cd39-4915-8b72-73209d7b6089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-cfb5b9ef-cd39-4915-8b72-73209d7b6089 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:32:58.228682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:32:58.229783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:32:58.230843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:32:58.232631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 190/371] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:32:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:32:58.239108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5b9ef-cd39-4915-8b72-73209d7b6089 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:58.250084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30927fb1-845f-4145-aff9-9c8adc04cb72 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:58.269500 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:58.327105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3893b14-4326-483a-933c-3992a502abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:58.329530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3893b14-4326-483a-933c-3992a502abf9 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:58.329963 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3893b14-4326-483a-933c-3992a502abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:58.330385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3893b14-4326-483a-933c-3992a502abf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:58.354609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:58.456858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5b9ef-cd39-4915-8b72-73209d7b6089 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:58.486057 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3893b14-4326-483a-933c-3992a502abf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:58.490127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3893b14-4326-483a-933c-3992a502abf9 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:58.512328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8033e0-445f-48e7-8d1e-18b861660e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:58.513227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8033e0-445f-48e7-8d1e-18b861660e5d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:58.513678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8033e0-445f-48e7-8d1e-18b861660e5d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:58.514096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8033e0-445f-48e7-8d1e-18b861660e5d None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:58.514974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8033e0-445f-48e7-8d1e-18b861660e5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:58.515475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8033e0-445f-48e7-8d1e-18b861660e5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:58.517014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8033e0-445f-48e7-8d1e-18b861660e5d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:58.527384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 191/372] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:32:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:32:58.533231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8033e0-445f-48e7-8d1e-18b861660e5d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:58.662103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:58.667468 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 182/373] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:32:58.672963 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3893b14-4326-483a-933c-3992a502abf9 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:58.686234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8033e0-445f-48e7-8d1e-18b861660e5d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:58.712138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:58.761798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947f20e-2ffc-4b1d-bddf-ffb7f6f43929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:58.762431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947f20e-2ffc-4b1d-bddf-ffb7f6f43929 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:58.762716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947f20e-2ffc-4b1d-bddf-ffb7f6f43929 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:58.762993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947f20e-2ffc-4b1d-bddf-ffb7f6f43929 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:58.859598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3893b14-4326-483a-933c-3992a502abf9 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:58.909150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947f20e-2ffc-4b1d-bddf-ffb7f6f43929 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:58.910952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947f20e-2ffc-4b1d-bddf-ffb7f6f43929 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:58.927807 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561972f-34dc-43bf-9426-aedc81739d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:58.928832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561972f-34dc-43bf-9426-aedc81739d82 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:58.929381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561972f-34dc-43bf-9426-aedc81739d82 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:58.929887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561972f-34dc-43bf-9426-aedc81739d82 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:58.932650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561972f-34dc-43bf-9426-aedc81739d82 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:58.935131 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561972f-34dc-43bf-9426-aedc81739d82 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:58.939805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 183/374] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:58 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:58.943817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561972f-34dc-43bf-9426-aedc81739d82 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:59.027374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:59.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561972f-34dc-43bf-9426-aedc81739d82 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:59.096509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 192/375] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:32:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 18:32:59.102608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947f20e-2ffc-4b1d-bddf-ffb7f6f43929 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:59.114753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12590edc-8062-4cd8-b167-05558ac58dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:59.115509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12590edc-8062-4cd8-b167-05558ac58dc1 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:59.115935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12590edc-8062-4cd8-b167-05558ac58dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:59.116311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12590edc-8062-4cd8-b167-05558ac58dc1 None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:59.118290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12590edc-8062-4cd8-b167-05558ac58dc1 None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:59.119944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12590edc-8062-4cd8-b167-05558ac58dc1 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:59.122036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 184/376] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:32:58 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:59.126213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:59.128784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12590edc-8062-4cd8-b167-05558ac58dc1 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:59.249958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947f20e-2ffc-4b1d-bddf-ffb7f6f43929 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:59.289331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7aa34-8e8b-492a-a7b9-6c5162f4da9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:59.289921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7aa34-8e8b-492a-a7b9-6c5162f4da9b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:59.290259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7aa34-8e8b-492a-a7b9-6c5162f4da9b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:59.290535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7aa34-8e8b-492a-a7b9-6c5162f4da9b None admin] PATH_INFO: `/v3/projects/1a0304ec85d64b64aa7cbc3f8a24053a` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:59.291951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7aa34-8e8b-492a-a7b9-6c5162f4da9b None admin] RBAC: Authorizing `identity:get_project(project_id=1a0304ec85d64b64aa7cbc3f8a24053a)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:59.293389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7aa34-8e8b-492a-a7b9-6c5162f4da9b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:59.295145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 193/377] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:32:59 2022] GET /identity/v3/projects/1a0304ec85d64b64aa7cbc3f8a24053a => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:59.302260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7aa34-8e8b-492a-a7b9-6c5162f4da9b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:59.315907 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12590edc-8062-4cd8-b167-05558ac58dc1 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:59.364151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeab436-f37c-485b-9359-729b0f0bcc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:59.365238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeab436-f37c-485b-9359-729b0f0bcc4d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:59.365726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeab436-f37c-485b-9359-729b0f0bcc4d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:59.366203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeab436-f37c-485b-9359-729b0f0bcc4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:59.367867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeab436-f37c-485b-9359-729b0f0bcc4d None admin] RBAC: Adding query filter params (group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role.id=6b3a5d54d6044e9b8d8270e7ff3145e8, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:59.368469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeab436-f37c-485b-9359-729b0f0bcc4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:59.370178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeab436-f37c-485b-9359-729b0f0bcc4d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:59.373776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:59.377372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 185/378] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 18:32:59 2022] GET /identity/v3/role_assignments?group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:59.430042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:32:59.451780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7aa34-8e8b-492a-a7b9-6c5162f4da9b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:59.489674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b795383-1ac4-46a0-a006-3513e30d37ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:59.490443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795383-1ac4-46a0-a006-3513e30d37ac None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:59.490697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795383-1ac4-46a0-a006-3513e30d37ac None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:59.490971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795383-1ac4-46a0-a006-3513e30d37ac None admin] PATH_INFO: `/v3/groups/8e4c593ada3d4c20a62fefe0322524f6` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:59.498736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b795383-1ac4-46a0-a006-3513e30d37ac None admin] RBAC: Authorizing `identity:get_group(group_id=8e4c593ada3d4c20a62fefe0322524f6)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:59.500263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b795383-1ac4-46a0-a006-3513e30d37ac None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:59.502347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 194/379] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:32:59 2022] GET /identity/v3/groups/8e4c593ada3d4c20a62fefe0322524f6 => generated 229 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:32:59.509257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeab436-f37c-485b-9359-729b0f0bcc4d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:32:59.651317 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeab436-f37c-485b-9359-729b0f0bcc4d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:32:59.687423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba2b01-d77e-4e7d-9214-cd1570981870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:32:59.688388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba2b01-d77e-4e7d-9214-cd1570981870 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:32:59.688890 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba2b01-d77e-4e7d-9214-cd1570981870 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:32:59.689393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba2b01-d77e-4e7d-9214-cd1570981870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:32:59.690319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba2b01-d77e-4e7d-9214-cd1570981870 None admin] RBAC: Adding query filter params (group.id=8e4c593ada3d4c20a62fefe0322524f6, role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=1a0304ec85d64b64aa7cbc3f8a24053a) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:32:59.690854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba2b01-d77e-4e7d-9214-cd1570981870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:32:59.692497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba2b01-d77e-4e7d-9214-cd1570981870 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:32:59.699118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 186/380] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 18:32:59 2022] GET /identity/v3/role_assignments?group.id=8e4c593ada3d4c20a62fefe0322524f6&scope.project.id=1a0304ec85d64b64aa7cbc3f8a24053a&role.id=5c777de40b8449329d714d8c2229115e => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:00.591705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156692-1d58-4c88-ae73-db1d46e6a52e None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:00.591705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156692-1d58-4c88-ae73-db1d46e6a52e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:00.591705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2156692-1d58-4c88-ae73-db1d46e6a52e None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:00.592791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 195/381] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:00.600189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34e644-c116-4707-b82e-8094813bade8 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:00.601077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34e644-c116-4707-b82e-8094813bade8 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:00.601677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34e644-c116-4707-b82e-8094813bade8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:00.630582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-de34e644-c116-4707-b82e-8094813bade8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:00.631807 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-de34e644-c116-4707-b82e-8094813bade8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:00.646017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f8850-d824-4949-8d20-b18f1d7eacf5 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:00.646335 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f8850-d824-4949-8d20-b18f1d7eacf5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:00.648082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f8850-d824-4949-8d20-b18f1d7eacf5 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:00.650612 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 196/382] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:00.658030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cca41-a1a5-41a1-b885-f2964089f2a1 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:00.658329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cca41-a1a5-41a1-b885-f2964089f2a1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:00.658587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cca41-a1a5-41a1-b885-f2964089f2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:00.688699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e14cca41-a1a5-41a1-b885-f2964089f2a1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:00.689953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e14cca41-a1a5-41a1-b885-f2964089f2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:00.757412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:00.772610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 187/383] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:33:00.806132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 197/384] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:33:00.830639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964982d-b667-4417-a4aa-14826dd9bc02 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:00.830937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964982d-b667-4417-a4aa-14826dd9bc02 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:00.831202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964982d-b667-4417-a4aa-14826dd9bc02 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:00.832576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 188/385] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:00.838019 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966bb77-2e3c-4969-9f0b-1a037bd038da None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:00.838310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966bb77-2e3c-4969-9f0b-1a037bd038da None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:00.838587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966bb77-2e3c-4969-9f0b-1a037bd038da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:00.860632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-3966bb77-2e3c-4969-9f0b-1a037bd038da None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:00.861788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3966bb77-2e3c-4969-9f0b-1a037bd038da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:00.959493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 198/386] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:01.082567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964982d-b667-4417-a4aa-14826dd9bc02 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:01.178718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966bb77-2e3c-4969-9f0b-1a037bd038da None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:01.201687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:01.212989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964982d-b667-4417-a4aa-14826dd9bc02 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:01.257207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490b44c-4193-4f0e-9e78-bab040494f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:01.257925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490b44c-4193-4f0e-9e78-bab040494f06 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:01.258206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490b44c-4193-4f0e-9e78-bab040494f06 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:01.258706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490b44c-4193-4f0e-9e78-bab040494f06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:01.282225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490b44c-4193-4f0e-9e78-bab040494f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-3490b44c-4193-4f0e-9e78-bab040494f06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:01.291812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:01.292694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:33:01.293536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:01.294183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 189/387] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:01.300979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490b44c-4193-4f0e-9e78-bab040494f06 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:01.361171 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966bb77-2e3c-4969-9f0b-1a037bd038da None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:01.376693 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:01.400861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93dfd6-49af-4be7-8f0d-e07555b31b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:01.401684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93dfd6-49af-4be7-8f0d-e07555b31b6a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:01.402041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93dfd6-49af-4be7-8f0d-e07555b31b6a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:01.402395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93dfd6-49af-4be7-8f0d-e07555b31b6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:01.415185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93dfd6-49af-4be7-8f0d-e07555b31b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-1d93dfd6-49af-4be7-8f0d-e07555b31b6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:01.420578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:01.421518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:01.422362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 199/388] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:33:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:01.423012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93dfd6-49af-4be7-8f0d-e07555b31b6a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:01.439325 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490b44c-4193-4f0e-9e78-bab040494f06 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:01.481767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c0d46-3573-44d6-b2ca-15bd460a1f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:01.482891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c0d46-3573-44d6-b2ca-15bd460a1f05 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:01.483473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c0d46-3573-44d6-b2ca-15bd460a1f05 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:01.484057 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c0d46-3573-44d6-b2ca-15bd460a1f05 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:01.485121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c0d46-3573-44d6-b2ca-15bd460a1f05 None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:01.485633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c0d46-3573-44d6-b2ca-15bd460a1f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:01.487439 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c0d46-3573-44d6-b2ca-15bd460a1f05 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:01.498226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 190/389] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:33:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:01.499473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c0d46-3573-44d6-b2ca-15bd460a1f05 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:01.512272 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:01.556766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93dfd6-49af-4be7-8f0d-e07555b31b6a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:01.599737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df850db-7886-499a-aac3-41f2c0f6158d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:01.600509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df850db-7886-499a-aac3-41f2c0f6158d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:01.600681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df850db-7886-499a-aac3-41f2c0f6158d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:01.600936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df850db-7886-499a-aac3-41f2c0f6158d None admin] PATH_INFO: `/v3/roles/member` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:01.615555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df850db-7886-499a-aac3-41f2c0f6158d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-8df850db-7886-499a-aac3-41f2c0f6158d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:01.621007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:01.622086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 200/390] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:01.623871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df850db-7886-499a-aac3-41f2c0f6158d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:01.649235 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c0d46-3573-44d6-b2ca-15bd460a1f05 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:01.670735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:01.695515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46736394-dc6a-4dd3-ba8f-d8e0c0079717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:01.695515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46736394-dc6a-4dd3-ba8f-d8e0c0079717 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:01.695515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46736394-dc6a-4dd3-ba8f-d8e0c0079717 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:01.695515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46736394-dc6a-4dd3-ba8f-d8e0c0079717 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:01.696175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46736394-dc6a-4dd3-ba8f-d8e0c0079717 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:01.696175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46736394-dc6a-4dd3-ba8f-d8e0c0079717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:01.697575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46736394-dc6a-4dd3-ba8f-d8e0c0079717 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:01.711981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 191/391] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:01.712876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46736394-dc6a-4dd3-ba8f-d8e0c0079717 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:01.770520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df850db-7886-499a-aac3-41f2c0f6158d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:01.804927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:01.815258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2c97a-1cbe-40cd-8f94-f6227c0d92ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:01.819548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2c97a-1cbe-40cd-8f94-f6227c0d92ac None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:01.819548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2c97a-1cbe-40cd-8f94-f6227c0d92ac None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:01.819548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2c97a-1cbe-40cd-8f94-f6227c0d92ac None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:01.819548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2c97a-1cbe-40cd-8f94-f6227c0d92ac None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:01.821454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2c97a-1cbe-40cd-8f94-f6227c0d92ac None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:01.828430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 201/392] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:33:01 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:01.828430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2c97a-1cbe-40cd-8f94-f6227c0d92ac None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:01.831677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46736394-dc6a-4dd3-ba8f-d8e0c0079717 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:01.870448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e81d96-874a-40b9-89da-0ad73c154fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:01.871240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e81d96-874a-40b9-89da-0ad73c154fd6 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:01.871644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e81d96-874a-40b9-89da-0ad73c154fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:01.872022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e81d96-874a-40b9-89da-0ad73c154fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:01.873078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e81d96-874a-40b9-89da-0ad73c154fd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:01.873211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e81d96-874a-40b9-89da-0ad73c154fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:01.874734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e81d96-874a-40b9-89da-0ad73c154fd6 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:01.884270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 192/393] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:33:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:01.885426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e81d96-874a-40b9-89da-0ad73c154fd6 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:01.936854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:01.963258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2c97a-1cbe-40cd-8f94-f6227c0d92ac None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:02.003649 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e81d96-874a-40b9-89da-0ad73c154fd6 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:02.006279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921f9f4-aec8-4391-b70b-d64d5d49e0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:02.007111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921f9f4-aec8-4391-b70b-d64d5d49e0a7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.007634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921f9f4-aec8-4391-b70b-d64d5d49e0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.008068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921f9f4-aec8-4391-b70b-d64d5d49e0a7 None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.009389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921f9f4-aec8-4391-b70b-d64d5d49e0a7 None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:02.010986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921f9f4-aec8-4391-b70b-d64d5d49e0a7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:02.013153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 202/394] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:33:01 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:02.014080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921f9f4-aec8-4391-b70b-d64d5d49e0a7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:02.043270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c29d77-8fdf-4e7b-8718-40e70dcb7033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:02.043990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29d77-8fdf-4e7b-8718-40e70dcb7033 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.044340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29d77-8fdf-4e7b-8718-40e70dcb7033 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.044687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29d77-8fdf-4e7b-8718-40e70dcb7033 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.045929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c29d77-8fdf-4e7b-8718-40e70dcb7033 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:02.047412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c29d77-8fdf-4e7b-8718-40e70dcb7033 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:02.053170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 193/395] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:33:01 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:02.053170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426004a-c7f0-42a9-be3d-6303803b1b6b None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.053170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426004a-c7f0-42a9-be3d-6303803b1b6b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.053170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426004a-c7f0-42a9-be3d-6303803b1b6b None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.053170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 194/396] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:02.053170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426004a-c7f0-42a9-be3d-6303803b1b6b None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:02.094442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:02.134518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921f9f4-aec8-4391-b70b-d64d5d49e0a7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:02.160451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426004a-c7f0-42a9-be3d-6303803b1b6b None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:02.170170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:02.173547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81138130-f014-41ef-890c-8f8113088aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:02.174190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81138130-f014-41ef-890c-8f8113088aee None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.174473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81138130-f014-41ef-890c-8f8113088aee None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.174741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81138130-f014-41ef-890c-8f8113088aee None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.176038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81138130-f014-41ef-890c-8f8113088aee None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:02.177505 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81138130-f014-41ef-890c-8f8113088aee None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:02.184274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 203/397] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:33:02 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:02.184274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81138130-f014-41ef-890c-8f8113088aee None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:02.195850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6f4f0-a5af-4e94-a063-d1df3b9a4090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:02.196493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f4f0-a5af-4e94-a063-d1df3b9a4090 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.196785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f4f0-a5af-4e94-a063-d1df3b9a4090 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.197073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f4f0-a5af-4e94-a063-d1df3b9a4090 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.198258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6f4f0-a5af-4e94-a063-d1df3b9a4090 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:02.199737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6f4f0-a5af-4e94-a063-d1df3b9a4090 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:02.207095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 195/398] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:33:02 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:02.208429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59e157-aec4-4505-afeb-d29a8af85984 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.208723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59e157-aec4-4505-afeb-d29a8af85984 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.209000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59e157-aec4-4505-afeb-d29a8af85984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.229423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-7f59e157-aec4-4505-afeb-d29a8af85984 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:02.230585 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7f59e157-aec4-4505-afeb-d29a8af85984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:02.302267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81138130-f014-41ef-890c-8f8113088aee None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:02.307057 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 196/399] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:02.308134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59e157-aec4-4505-afeb-d29a8af85984 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:02.334110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:02.341363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51177389-f5fd-4c25-8ab8-d955588e34d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:02.342115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51177389-f5fd-4c25-8ab8-d955588e34d6 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.342464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51177389-f5fd-4c25-8ab8-d955588e34d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.343007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51177389-f5fd-4c25-8ab8-d955588e34d6 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.344953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51177389-f5fd-4c25-8ab8-d955588e34d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c59969df6296450f916add4011ddf885, group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:02.346621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51177389-f5fd-4c25-8ab8-d955588e34d6 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:02.421610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59e157-aec4-4505-afeb-d29a8af85984 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:02.429991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:02.445531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 204/400] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 8 18:33:02 2022] PUT /identity/v3/projects/c59969df6296450f916add4011ddf885/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:02.446692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51177389-f5fd-4c25-8ab8-d955588e34d6 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:02.463665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f22ce-fabe-4e43-9433-571652f28756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:02.465173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f22ce-fabe-4e43-9433-571652f28756 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.465467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f22ce-fabe-4e43-9433-571652f28756 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.465746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f22ce-fabe-4e43-9433-571652f28756 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.467242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f22ce-fabe-4e43-9433-571652f28756 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:02.469888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f22ce-fabe-4e43-9433-571652f28756 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:02.477136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 197/401] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:33:02 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:02.477136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a8c16-301f-4bcf-98c9-565a966a5fdb None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.477136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a8c16-301f-4bcf-98c9-565a966a5fdb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.477136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a8c16-301f-4bcf-98c9-565a966a5fdb None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.481434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 198/402] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:02.483187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a8c16-301f-4bcf-98c9-565a966a5fdb None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:02.605206 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51177389-f5fd-4c25-8ab8-d955588e34d6 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:02.629617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a8c16-301f-4bcf-98c9-565a966a5fdb None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:02.647551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb0d36-92a2-4711-b5d4-48f27ea1e6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:02.649084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb0d36-92a2-4711-b5d4-48f27ea1e6a8 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.649392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb0d36-92a2-4711-b5d4-48f27ea1e6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.649670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb0d36-92a2-4711-b5d4-48f27ea1e6a8 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d/roles/6b3a5d54d6044e9b8d8270e7ff3145e8` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.651939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb0d36-92a2-4711-b5d4-48f27ea1e6a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c59969df6296450f916add4011ddf885, group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role_id=6b3a5d54d6044e9b8d8270e7ff3145e8)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:02.653396 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb0d36-92a2-4711-b5d4-48f27ea1e6a8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:02.665684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39b1f3-4305-4d57-a4a2-c3d84685638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:02.671968 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39b1f3-4305-4d57-a4a2-c3d84685638b None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.671968 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39b1f3-4305-4d57-a4a2-c3d84685638b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.671968 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39b1f3-4305-4d57-a4a2-c3d84685638b None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.671968 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39b1f3-4305-4d57-a4a2-c3d84685638b None admin] RBAC: Authorizing `identity:create_grant(project_id=9ef8ff3249dd4e84ad8692c06a441377, group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:02.671968 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39b1f3-4305-4d57-a4a2-c3d84685638b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:02.731140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 205/403] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 8 18:33:02 2022] PUT /identity/v3/projects/c59969df6296450f916add4011ddf885/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d/roles/6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:02.732446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb6861-f392-4e8d-9c73-5c0f86ad9ec4 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.732673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb6861-f392-4e8d-9c73-5c0f86ad9ec4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.732944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb6861-f392-4e8d-9c73-5c0f86ad9ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:02.753666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-0cfb6861-f392-4e8d-9c73-5c0f86ad9ec4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:02.754888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0cfb6861-f392-4e8d-9c73-5c0f86ad9ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:02.756380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 199/404] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 8 18:33:02 2022] PUT /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:02.758587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39b1f3-4305-4d57-a4a2-c3d84685638b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:02.813562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:02.883833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:02.933377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 206/405] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:02.944051 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39b1f3-4305-4d57-a4a2-c3d84685638b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:02.996323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352c9aa-2096-4dde-8c4a-5d5e973495e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:02.996892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352c9aa-2096-4dde-8c4a-5d5e973495e6 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:02.997204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352c9aa-2096-4dde-8c4a-5d5e973495e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:02.997474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352c9aa-2096-4dde-8c4a-5d5e973495e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:03.013225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352c9aa-2096-4dde-8c4a-5d5e973495e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-1352c9aa-2096-4dde-8c4a-5d5e973495e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:03.018110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:03.019103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:03.020097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 200/406] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:33:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:03.025276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb6861-f392-4e8d-9c73-5c0f86ad9ec4 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:03.146185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb6861-f392-4e8d-9c73-5c0f86ad9ec4 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:03.191155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e21ab4-9e7e-423d-9319-eb5901303851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:03.191874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e21ab4-9e7e-423d-9319-eb5901303851 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:03.192155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e21ab4-9e7e-423d-9319-eb5901303851 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:03.192421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e21ab4-9e7e-423d-9319-eb5901303851 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:03.193118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e21ab4-9e7e-423d-9319-eb5901303851 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:03.193419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e21ab4-9e7e-423d-9319-eb5901303851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:03.194837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e21ab4-9e7e-423d-9319-eb5901303851 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:03.207247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 207/407] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:03.212995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352c9aa-2096-4dde-8c4a-5d5e973495e6 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:03.282409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e21ab4-9e7e-423d-9319-eb5901303851 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:03.339583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:03.347823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352c9aa-2096-4dde-8c4a-5d5e973495e6 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:03.379044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:03.391564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c967c-eb8e-469d-9992-47e8c67115f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:03.392520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c967c-eb8e-469d-9992-47e8c67115f1 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:03.392834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c967c-eb8e-469d-9992-47e8c67115f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:03.393083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c967c-eb8e-469d-9992-47e8c67115f1 None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:03.394314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c967c-eb8e-469d-9992-47e8c67115f1 None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:03.395849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c967c-eb8e-469d-9992-47e8c67115f1 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:03.397485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 201/408] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:33:03 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:03.402912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c967c-eb8e-469d-9992-47e8c67115f1 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:03.452176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e21ab4-9e7e-423d-9319-eb5901303851 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:03.523640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960c04e-b4a7-4cef-a705-858820e49191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:03.524675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960c04e-b4a7-4cef-a705-858820e49191 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:03.524970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960c04e-b4a7-4cef-a705-858820e49191 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:03.525287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960c04e-b4a7-4cef-a705-858820e49191 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:03.536477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960c04e-b4a7-4cef-a705-858820e49191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-e960c04e-b4a7-4cef-a705-858820e49191 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:03.542466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:03.543360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:03.544218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:03.548978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 208/409] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:03.549842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960c04e-b4a7-4cef-a705-858820e49191 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:03.575099 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c967c-eb8e-469d-9992-47e8c67115f1 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:03.655402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0808513-bdc2-478a-977f-81c6bd456a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:03.656041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0808513-bdc2-478a-977f-81c6bd456a4b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:03.656322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0808513-bdc2-478a-977f-81c6bd456a4b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:03.656597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0808513-bdc2-478a-977f-81c6bd456a4b None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:03.657747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0808513-bdc2-478a-977f-81c6bd456a4b None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:03.659183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0808513-bdc2-478a-977f-81c6bd456a4b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:03.660385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960c04e-b4a7-4cef-a705-858820e49191 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:03.665250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 202/410] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:33:03 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:03.666915 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0808513-bdc2-478a-977f-81c6bd456a4b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:03.689798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:03.708425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:03.714233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9e58d-e406-43f1-a968-65ca2387827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:03.715466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9e58d-e406-43f1-a968-65ca2387827b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:03.715799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9e58d-e406-43f1-a968-65ca2387827b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:03.716076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9e58d-e406-43f1-a968-65ca2387827b None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:03.716754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9e58d-e406-43f1-a968-65ca2387827b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:03.717076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9e58d-e406-43f1-a968-65ca2387827b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:03.723290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9e58d-e406-43f1-a968-65ca2387827b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:03.740632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 209/411] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:03.747228 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9e58d-e406-43f1-a968-65ca2387827b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:03.828095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0808513-bdc2-478a-977f-81c6bd456a4b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:03.873745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9e58d-e406-43f1-a968-65ca2387827b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:03.907908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6ef7-214c-498d-a760-3e5edeafa0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:03.908582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6ef7-214c-498d-a760-3e5edeafa0a3 None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:03.908851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6ef7-214c-498d-a760-3e5edeafa0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:03.909149 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6ef7-214c-498d-a760-3e5edeafa0a3 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d/roles/6b3a5d54d6044e9b8d8270e7ff3145e8` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:03.915946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f6ef7-214c-498d-a760-3e5edeafa0a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ef8ff3249dd4e84ad8692c06a441377, group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role_id=6b3a5d54d6044e9b8d8270e7ff3145e8)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:03.917447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f6ef7-214c-498d-a760-3e5edeafa0a3 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:03.924588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3548-d3dc-4c84-88b1-f982fc69af14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:03.925261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3548-d3dc-4c84-88b1-f982fc69af14 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:03.925596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3548-d3dc-4c84-88b1-f982fc69af14 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:03.925873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3548-d3dc-4c84-88b1-f982fc69af14 None admin] PATH_INFO: `/v3/groups/8e4c593ada3d4c20a62fefe0322524f6` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:03.927046 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3548-d3dc-4c84-88b1-f982fc69af14 None admin] RBAC: Authorizing `identity:get_group(group_id=8e4c593ada3d4c20a62fefe0322524f6)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:03.928548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3548-d3dc-4c84-88b1-f982fc69af14 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:03.931044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 210/412] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:33:03 2022] GET /identity/v3/groups/8e4c593ada3d4c20a62fefe0322524f6 => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:03.936190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3548-d3dc-4c84-88b1-f982fc69af14 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:03.970924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:04.010344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 203/413] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 8 18:33:03 2022] PUT /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d/roles/6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:33:04.051423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3548-d3dc-4c84-88b1-f982fc69af14 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:04.089607 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133098a-861d-4b59-8034-151b6029dfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:04.090176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133098a-861d-4b59-8034-151b6029dfc2 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:04.090465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133098a-861d-4b59-8034-151b6029dfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:04.090745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133098a-861d-4b59-8034-151b6029dfc2 None admin] PATH_INFO: `/v3/projects/1a0304ec85d64b64aa7cbc3f8a24053a` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:04.092011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133098a-861d-4b59-8034-151b6029dfc2 None admin] RBAC: Authorizing `identity:get_project(project_id=1a0304ec85d64b64aa7cbc3f8a24053a)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:04.093783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133098a-861d-4b59-8034-151b6029dfc2 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:04.095739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 211/414] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:33:03 2022] GET /identity/v3/projects/1a0304ec85d64b64aa7cbc3f8a24053a => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:04.102006 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6ef7-214c-498d-a760-3e5edeafa0a3 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:04.226800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:04.244115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6ef7-214c-498d-a760-3e5edeafa0a3 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:04.291498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502fc77-70f4-4155-8e90-686df67bbb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:04.292182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502fc77-70f4-4155-8e90-686df67bbb47 None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:04.292468 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502fc77-70f4-4155-8e90-686df67bbb47 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:04.292794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502fc77-70f4-4155-8e90-686df67bbb47 None admin] PATH_INFO: `/v3/projects/1a0304ec85d64b64aa7cbc3f8a24053a/groups/8e4c593ada3d4c20a62fefe0322524f6/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:04.295031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502fc77-70f4-4155-8e90-686df67bbb47 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a0304ec85d64b64aa7cbc3f8a24053a, group_id=8e4c593ada3d4c20a62fefe0322524f6, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:04.297379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502fc77-70f4-4155-8e90-686df67bbb47 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:04.394152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 204/415] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 8 18:33:04 2022] PUT /identity/v3/projects/1a0304ec85d64b64aa7cbc3f8a24053a/groups/8e4c593ada3d4c20a62fefe0322524f6/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:04.771930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d4617-51eb-4355-bb68-2098d6907842 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:04.772586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d4617-51eb-4355-bb68-2098d6907842 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:04.772586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d4617-51eb-4355-bb68-2098d6907842 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:04.774069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 212/416] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:04.779494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e07a6-e901-48f0-8656-16dd390f1989 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:04.779991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e07a6-e901-48f0-8656-16dd390f1989 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:04.780159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e07a6-e901-48f0-8656-16dd390f1989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:04.801839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b76e07a6-e901-48f0-8656-16dd390f1989 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:04.803040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b76e07a6-e901-48f0-8656-16dd390f1989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:04.889030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 205/417] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:05.327904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d4617-51eb-4355-bb68-2098d6907842 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:05.438142 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345813e-d02e-4c17-aa61-a0bc850ac550 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:05.438518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345813e-d02e-4c17-aa61-a0bc850ac550 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:05.438709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345813e-d02e-4c17-aa61-a0bc850ac550 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:05.440306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 206/418] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:05.455210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfce502-0490-4361-bc64-8212bd406c37 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:05.455508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfce502-0490-4361-bc64-8212bd406c37 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:05.455759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfce502-0490-4361-bc64-8212bd406c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:05.480043 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:05.487429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d4617-51eb-4355-bb68-2098d6907842 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:05.490434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-0bfce502-0490-4361-bc64-8212bd406c37 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:05.491627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0bfce502-0490-4361-bc64-8212bd406c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:05.529639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416c29e-6d76-4c7a-8c9c-ff40a4502354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:05.530285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c29e-6d76-4c7a-8c9c-ff40a4502354 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:05.530561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c29e-6d76-4c7a-8c9c-ff40a4502354 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:05.530829 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c29e-6d76-4c7a-8c9c-ff40a4502354 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:05.544169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416c29e-6d76-4c7a-8c9c-ff40a4502354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-7416c29e-6d76-4c7a-8c9c-ff40a4502354 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:05.549508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:05.550492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:05.551910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 213/419] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:05.562536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416c29e-6d76-4c7a-8c9c-ff40a4502354 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:05.623929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 207/420] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:33:05.682367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf144f97-ba8a-41c5-98b8-53e82c3b0753 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:05.682765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf144f97-ba8a-41c5-98b8-53e82c3b0753 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:05.683016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf144f97-ba8a-41c5-98b8-53e82c3b0753 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:05.684497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 208/421] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:33:05.689361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f93fa6-5cec-42a1-a1c1-5db64e6a4659 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:05.689645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f93fa6-5cec-42a1-a1c1-5db64e6a4659 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:05.689924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f93fa6-5cec-42a1-a1c1-5db64e6a4659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:05.719429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-64f93fa6-5cec-42a1-a1c1-5db64e6a4659 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:05.722219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-64f93fa6-5cec-42a1-a1c1-5db64e6a4659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:05.727952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:05.743744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416c29e-6d76-4c7a-8c9c-ff40a4502354 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:05.807595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087e7b7-d9aa-4d41-b1de-8c1ed6d6d6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:05.808263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087e7b7-d9aa-4d41-b1de-8c1ed6d6d6bf None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:05.808478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087e7b7-d9aa-4d41-b1de-8c1ed6d6d6bf None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:05.808746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087e7b7-d9aa-4d41-b1de-8c1ed6d6d6bf None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:05.809502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087e7b7-d9aa-4d41-b1de-8c1ed6d6d6bf None admin] RBAC: Adding query filter params (name=member) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:05.809809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087e7b7-d9aa-4d41-b1de-8c1ed6d6d6bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:05.811243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087e7b7-d9aa-4d41-b1de-8c1ed6d6d6bf None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:05.820344 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 214/422] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:33:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:05.825769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087e7b7-d9aa-4d41-b1de-8c1ed6d6d6bf None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:05.850184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 209/423] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:33:05.874102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:05.917427 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f93fa6-5cec-42a1-a1c1-5db64e6a4659 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:06.006016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087e7b7-d9aa-4d41-b1de-8c1ed6d6d6bf None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:06.059850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f93fa6-5cec-42a1-a1c1-5db64e6a4659 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:06.073440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e6702-3a69-434b-9bd7-cf3f2bd11505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:06.074353 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e6702-3a69-434b-9bd7-cf3f2bd11505 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:06.074856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e6702-3a69-434b-9bd7-cf3f2bd11505 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:06.075336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e6702-3a69-434b-9bd7-cf3f2bd11505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:06.125844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7881b-f562-499d-821a-bff45e1ae280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:06.126428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7881b-f562-499d-821a-bff45e1ae280 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:06.126667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7881b-f562-499d-821a-bff45e1ae280 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:06.126939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7881b-f562-499d-821a-bff45e1ae280 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:06.139342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:06.144680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7881b-f562-499d-821a-bff45e1ae280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-cae7881b-f562-499d-821a-bff45e1ae280 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:06.149759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:06.157960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:06.158737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 210/424] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:33:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:06.159416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e6702-3a69-434b-9bd7-cf3f2bd11505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:06.159416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e6702-3a69-434b-9bd7-cf3f2bd11505 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:06.171167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7881b-f562-499d-821a-bff45e1ae280 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:06.187282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:06.270092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 215/425] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 18:33:06.273481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e6702-3a69-434b-9bd7-cf3f2bd11505 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:06.325280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7881b-f562-499d-821a-bff45e1ae280 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:06.364647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b721d-59ff-46c8-b040-4269ccc23bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:06.365242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b721d-59ff-46c8-b040-4269ccc23bd8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:06.365589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b721d-59ff-46c8-b040-4269ccc23bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:06.365946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b721d-59ff-46c8-b040-4269ccc23bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:06.366676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b721d-59ff-46c8-b040-4269ccc23bd8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:06.367628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b721d-59ff-46c8-b040-4269ccc23bd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:06.370199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b721d-59ff-46c8-b040-4269ccc23bd8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:06.382244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e6702-3a69-434b-9bd7-cf3f2bd11505 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:06.388576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 211/426] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:06.388576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b721d-59ff-46c8-b040-4269ccc23bd8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:06.412477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:06.430097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd9b40-fd0b-4206-921d-7d86b5e2102a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:06.431140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd9b40-fd0b-4206-921d-7d86b5e2102a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:06.431579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd9b40-fd0b-4206-921d-7d86b5e2102a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:06.436237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd9b40-fd0b-4206-921d-7d86b5e2102a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:06.444480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd9b40-fd0b-4206-921d-7d86b5e2102a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-b8bd9b40-fd0b-4206-921d-7d86b5e2102a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:06.450693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:06.451570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 18:33:06.452359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:06.454830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 216/427] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:06.455925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd9b40-fd0b-4206-921d-7d86b5e2102a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:06.461414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:06.504452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b721d-59ff-46c8-b040-4269ccc23bd8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:06.541383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cd13a-afcb-435b-92b6-bf4e67bf1566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:06.542012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cd13a-afcb-435b-92b6-bf4e67bf1566 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:06.542291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cd13a-afcb-435b-92b6-bf4e67bf1566 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:06.542564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cd13a-afcb-435b-92b6-bf4e67bf1566 None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:06.543730 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cd13a-afcb-435b-92b6-bf4e67bf1566 None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:06.545345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cd13a-afcb-435b-92b6-bf4e67bf1566 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:06.547396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 212/428] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:33:06 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:06.548177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cd13a-afcb-435b-92b6-bf4e67bf1566 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:06.570369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd9b40-fd0b-4206-921d-7d86b5e2102a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:06.620089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9f828-46fb-40cb-899e-834caedb8c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:06.621014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f828-46fb-40cb-899e-834caedb8c0d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:06.621552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f828-46fb-40cb-899e-834caedb8c0d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:06.622021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f828-46fb-40cb-899e-834caedb8c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:06.656494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cd13a-afcb-435b-92b6-bf4e67bf1566 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:06.676561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:06.687912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:06.693644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d3bde-6ed8-4989-b97c-f717677ab616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:06.694252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3bde-6ed8-4989-b97c-f717677ab616 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:06.694529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3bde-6ed8-4989-b97c-f717677ab616 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:06.694826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3bde-6ed8-4989-b97c-f717677ab616 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:06.695724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d3bde-6ed8-4989-b97c-f717677ab616 None admin] RBAC: Adding query filter params (name=member) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:06.695997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d3bde-6ed8-4989-b97c-f717677ab616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:06.697478 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d3bde-6ed8-4989-b97c-f717677ab616 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:06.699617 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9f828-46fb-40cb-899e-834caedb8c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:06.701369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9f828-46fb-40cb-899e-834caedb8c0d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:06.711885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 213/429] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:33:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:06.712578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d3bde-6ed8-4989-b97c-f717677ab616 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:06.810060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 217/430] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:06.811612 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9f828-46fb-40cb-899e-834caedb8c0d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:06.827490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d3bde-6ed8-4989-b97c-f717677ab616 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:06.866419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe657109-6754-42c0-8c9c-49f9e0103b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:06.867283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe657109-6754-42c0-8c9c-49f9e0103b98 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:06.867598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe657109-6754-42c0-8c9c-49f9e0103b98 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:06.867874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe657109-6754-42c0-8c9c-49f9e0103b98 None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:06.869038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe657109-6754-42c0-8c9c-49f9e0103b98 None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:06.870620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe657109-6754-42c0-8c9c-49f9e0103b98 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:06.872492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 214/431] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:33:06 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:06.873298 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe657109-6754-42c0-8c9c-49f9e0103b98 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:06.901966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:06.921330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9f828-46fb-40cb-899e-834caedb8c0d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:06.939337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:06.958573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcfef4-cba1-46e7-bfa0-17156a95ae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:06.959238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcfef4-cba1-46e7-bfa0-17156a95ae6f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:06.959524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcfef4-cba1-46e7-bfa0-17156a95ae6f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:06.959818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcfef4-cba1-46e7-bfa0-17156a95ae6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:06.984399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe657109-6754-42c0-8c9c-49f9e0103b98 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:07.022853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba11404-9be4-40a7-a96d-1dacc8b7ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:07.024057 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba11404-9be4-40a7-a96d-1dacc8b7ef6e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.024338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba11404-9be4-40a7-a96d-1dacc8b7ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.024609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba11404-9be4-40a7-a96d-1dacc8b7ef6e None admin] PATH_INFO: `/v3/projects/c59969df6296450f916add4011ddf885` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.025771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba11404-9be4-40a7-a96d-1dacc8b7ef6e None admin] RBAC: Authorizing `identity:get_project(project_id=c59969df6296450f916add4011ddf885)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:07.027217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba11404-9be4-40a7-a96d-1dacc8b7ef6e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:07.028913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 215/432] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:33:06 2022] GET /identity/v3/projects/c59969df6296450f916add4011ddf885 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:07.029707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba11404-9be4-40a7-a96d-1dacc8b7ef6e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:07.040587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcfef4-cba1-46e7-bfa0-17156a95ae6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:07.042085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcfef4-cba1-46e7-bfa0-17156a95ae6f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:07.127375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:07.133716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba11404-9be4-40a7-a96d-1dacc8b7ef6e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:07.146389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 218/433] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:07.147703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073c0d8-353f-4dd2-9c2c-2b0df473eda1 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.147703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073c0d8-353f-4dd2-9c2c-2b0df473eda1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.147969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073c0d8-353f-4dd2-9c2c-2b0df473eda1 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.153088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 219/434] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:07.153088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073c0d8-353f-4dd2-9c2c-2b0df473eda1 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:07.163629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:07.171144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f236c-8547-43fc-a277-8e6e5ca76c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:07.193250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f236c-8547-43fc-a277-8e6e5ca76c28 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.193250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f236c-8547-43fc-a277-8e6e5ca76c28 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.193250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f236c-8547-43fc-a277-8e6e5ca76c28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.193250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f236c-8547-43fc-a277-8e6e5ca76c28 None admin] RBAC: Adding query filter params (group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role.id=eccc3f8721b1466b8a1968b77e5c464f, scope.project.id=c59969df6296450f916add4011ddf885) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:07.193250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f236c-8547-43fc-a277-8e6e5ca76c28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:07.193250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f236c-8547-43fc-a277-8e6e5ca76c28 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:07.193746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 216/435] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 18:33:07 2022] GET /identity/v3/role_assignments?group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d&scope.project.id=c59969df6296450f916add4011ddf885&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:07.193746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986551a8-9c0d-401a-bf59-4c3068af9467 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.193746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986551a8-9c0d-401a-bf59-4c3068af9467 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.193746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986551a8-9c0d-401a-bf59-4c3068af9467 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.193746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 217/436] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:07.193746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986551a8-9c0d-401a-bf59-4c3068af9467 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:07.267279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073c0d8-353f-4dd2-9c2c-2b0df473eda1 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:07.306118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff992f-4160-410a-a520-80bcc9911112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:07.306337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986551a8-9c0d-401a-bf59-4c3068af9467 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:07.306769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff992f-4160-410a-a520-80bcc9911112 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.307049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff992f-4160-410a-a520-80bcc9911112 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.307323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff992f-4160-410a-a520-80bcc9911112 None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.308494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff992f-4160-410a-a520-80bcc9911112 None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:07.310031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff992f-4160-410a-a520-80bcc9911112 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:07.316954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 220/437] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:33:07 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:07.316954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10077d68-3dc2-4e8a-9c2c-da1175bd8d6e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.316954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10077d68-3dc2-4e8a-9c2c-da1175bd8d6e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.316954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10077d68-3dc2-4e8a-9c2c-da1175bd8d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.335256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-10077d68-3dc2-4e8a-9c2c-da1175bd8d6e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:07.337551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-10077d68-3dc2-4e8a-9c2c-da1175bd8d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:07.348332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1b4d9-fda0-4e20-831a-026c134eec39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:07.349146 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1b4d9-fda0-4e20-831a-026c134eec39 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.349428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1b4d9-fda0-4e20-831a-026c134eec39 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.349701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1b4d9-fda0-4e20-831a-026c134eec39 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.351001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1b4d9-fda0-4e20-831a-026c134eec39 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:07.352498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1b4d9-fda0-4e20-831a-026c134eec39 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:07.354344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 218/438] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:33:07 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:07.355285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48af7c-bc87-4e49-8877-799c89bb8170 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.355645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48af7c-bc87-4e49-8877-799c89bb8170 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.356546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48af7c-bc87-4e49-8877-799c89bb8170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.376684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-bd48af7c-bc87-4e49-8877-799c89bb8170 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:07.377938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bd48af7c-bc87-4e49-8877-799c89bb8170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:07.402120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:07.431513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 221/439] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:07.431810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10077d68-3dc2-4e8a-9c2c-da1175bd8d6e None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:07.447663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:07.469074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 219/440] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:07.469074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48af7c-bc87-4e49-8877-799c89bb8170 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:07.565952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10077d68-3dc2-4e8a-9c2c-da1175bd8d6e None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:07.571433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48af7c-bc87-4e49-8877-799c89bb8170 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:07.619414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4c144-6705-4c9b-8d7b-f75f784fc25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:07.619925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509c54f-a0c0-4b52-aa02-fc8f0291d776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:07.620538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509c54f-a0c0-4b52-aa02-fc8f0291d776 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.620820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509c54f-a0c0-4b52-aa02-fc8f0291d776 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.620929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4c144-6705-4c9b-8d7b-f75f784fc25c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.621175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509c54f-a0c0-4b52-aa02-fc8f0291d776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.621651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4c144-6705-4c9b-8d7b-f75f784fc25c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.621865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509c54f-a0c0-4b52-aa02-fc8f0291d776 None admin] RBAC: Adding query filter params (group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role.id=6b3a5d54d6044e9b8d8270e7ff3145e8, scope.project.id=c59969df6296450f916add4011ddf885) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:07.622180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509c54f-a0c0-4b52-aa02-fc8f0291d776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:07.622539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4c144-6705-4c9b-8d7b-f75f784fc25c None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.623712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509c54f-a0c0-4b52-aa02-fc8f0291d776 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:07.624082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4c144-6705-4c9b-8d7b-f75f784fc25c None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:07.625745 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4c144-6705-4c9b-8d7b-f75f784fc25c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:07.627897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 220/441] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:33:07 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:07.629946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 222/442] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 18:33:07 2022] GET /identity/v3/role_assignments?group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d&scope.project.id=c59969df6296450f916add4011ddf885&role.id=6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:07.638518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4c144-6705-4c9b-8d7b-f75f784fc25c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:07.709515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:07.738134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509c54f-a0c0-4b52-aa02-fc8f0291d776 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:07.763891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4c144-6705-4c9b-8d7b-f75f784fc25c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:07.796447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:07.808365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2123357-4940-4273-893e-b31eae45b6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:07.808984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2123357-4940-4273-893e-b31eae45b6d0 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.810267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2123357-4940-4273-893e-b31eae45b6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.810267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2123357-4940-4273-893e-b31eae45b6d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.810267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2123357-4940-4273-893e-b31eae45b6d0 None admin] RBAC: Adding query filter params (group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role.id=eccc3f8721b1466b8a1968b77e5c464f, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:07.810571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2123357-4940-4273-893e-b31eae45b6d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:07.812045 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2123357-4940-4273-893e-b31eae45b6d0 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:07.818775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 221/443] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 18:33:07 2022] GET /identity/v3/role_assignments?group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=eccc3f8721b1466b8a1968b77e5c464f => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:07.819638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2123357-4940-4273-893e-b31eae45b6d0 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:07.882833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509c54f-a0c0-4b52-aa02-fc8f0291d776 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:07.942040 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7cadd-e659-4842-a37f-0988992a5fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:07.942867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7cadd-e659-4842-a37f-0988992a5fc9 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:07.943155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7cadd-e659-4842-a37f-0988992a5fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:07.943434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7cadd-e659-4842-a37f-0988992a5fc9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:07.961884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7cadd-e659-4842-a37f-0988992a5fc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-12d7cadd-e659-4842-a37f-0988992a5fc9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:07.968646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:07.969684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:33:07.970668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:07.971535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2123357-4940-4273-893e-b31eae45b6d0 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:07.972088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 223/444] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:33:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:07.976787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7cadd-e659-4842-a37f-0988992a5fc9 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:08.010858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:08.031294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dd8e2-e870-4924-a14e-ab0e6f326503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:08.031979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dd8e2-e870-4924-a14e-ab0e6f326503 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:08.032260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dd8e2-e870-4924-a14e-ab0e6f326503 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:08.032535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dd8e2-e870-4924-a14e-ab0e6f326503 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:08.048487 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dd8e2-e870-4924-a14e-ab0e6f326503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-ea0dd8e2-e870-4924-a14e-ab0e6f326503 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:08.055062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:08.055994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:08.057020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 222/445] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:08.063065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dd8e2-e870-4924-a14e-ab0e6f326503 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:08.118509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7cadd-e659-4842-a37f-0988992a5fc9 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:08.121593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:08.160042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18b3a2-5a72-4c02-a840-cbf53c7f88b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:08.160654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18b3a2-5a72-4c02-a840-cbf53c7f88b9 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:08.160929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18b3a2-5a72-4c02-a840-cbf53c7f88b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:08.161228 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18b3a2-5a72-4c02-a840-cbf53c7f88b9 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:08.161897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18b3a2-5a72-4c02-a840-cbf53c7f88b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:08.162197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18b3a2-5a72-4c02-a840-cbf53c7f88b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:08.163576 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18b3a2-5a72-4c02-a840-cbf53c7f88b9 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:08.172928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 224/446] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:08.180539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18b3a2-5a72-4c02-a840-cbf53c7f88b9 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:08.189464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dd8e2-e870-4924-a14e-ab0e6f326503 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:08.231022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a21f21-69f4-46d0-a2bb-9237a44e4fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:08.231555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a21f21-69f4-46d0-a2bb-9237a44e4fef None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:08.231841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a21f21-69f4-46d0-a2bb-9237a44e4fef None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:08.232117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a21f21-69f4-46d0-a2bb-9237a44e4fef None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:08.232792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a21f21-69f4-46d0-a2bb-9237a44e4fef None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:08.233229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a21f21-69f4-46d0-a2bb-9237a44e4fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:08.234636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a21f21-69f4-46d0-a2bb-9237a44e4fef None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:08.244040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 223/447] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:08.250432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a21f21-69f4-46d0-a2bb-9237a44e4fef None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:08.308241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:08.309022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18b3a2-5a72-4c02-a840-cbf53c7f88b9 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:08.343909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274522f-3d57-4c87-be3d-1f656ce167a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:08.344523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274522f-3d57-4c87-be3d-1f656ce167a4 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:08.344806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274522f-3d57-4c87-be3d-1f656ce167a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:08.345117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274522f-3d57-4c87-be3d-1f656ce167a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:08.363241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a21f21-69f4-46d0-a2bb-9237a44e4fef None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:08.379717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:08.396797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6e498-d3a7-4e2c-bad6-55fe2241eac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:08.397438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6e498-d3a7-4e2c-bad6-55fe2241eac8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:08.397720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6e498-d3a7-4e2c-bad6-55fe2241eac8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:08.398166 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6e498-d3a7-4e2c-bad6-55fe2241eac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:08.419109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274522f-3d57-4c87-be3d-1f656ce167a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:08.420611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274522f-3d57-4c87-be3d-1f656ce167a4 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:08.471861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6e498-d3a7-4e2c-bad6-55fe2241eac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:08.473357 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6e498-d3a7-4e2c-bad6-55fe2241eac8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:08.520758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:08.528249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 225/448] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 18:33:08.529922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274522f-3d57-4c87-be3d-1f656ce167a4 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:08.576857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 224/449] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 18:33:08.582530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6e498-d3a7-4e2c-bad6-55fe2241eac8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:08.597298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:08.632648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274522f-3d57-4c87-be3d-1f656ce167a4 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:08.666891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f15f8-fee7-49fc-b360-b30ed4e6e241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:08.667522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f15f8-fee7-49fc-b360-b30ed4e6e241 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:08.667799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f15f8-fee7-49fc-b360-b30ed4e6e241 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:08.668071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f15f8-fee7-49fc-b360-b30ed4e6e241 None admin] PATH_INFO: `/v3/projects/9ef8ff3249dd4e84ad8692c06a441377` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:08.669234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f15f8-fee7-49fc-b360-b30ed4e6e241 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef8ff3249dd4e84ad8692c06a441377)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:08.670667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f15f8-fee7-49fc-b360-b30ed4e6e241 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:08.672276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 226/450] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:33:08 2022] GET /identity/v3/projects/9ef8ff3249dd4e84ad8692c06a441377 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:08.677696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f15f8-fee7-49fc-b360-b30ed4e6e241 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:08.681803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6e498-d3a7-4e2c-bad6-55fe2241eac8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:08.715990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0cd08-132e-4c87-8d76-2a3d09e177b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:08.716615 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0cd08-132e-4c87-8d76-2a3d09e177b4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:08.716898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0cd08-132e-4c87-8d76-2a3d09e177b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:08.717216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0cd08-132e-4c87-8d76-2a3d09e177b4 None admin] PATH_INFO: `/v3/projects/1a0304ec85d64b64aa7cbc3f8a24053a` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:08.718606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0cd08-132e-4c87-8d76-2a3d09e177b4 None admin] RBAC: Authorizing `identity:get_project(project_id=1a0304ec85d64b64aa7cbc3f8a24053a)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:08.720034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0cd08-132e-4c87-8d76-2a3d09e177b4 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:08.721691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 225/451] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:33:08 2022] GET /identity/v3/projects/1a0304ec85d64b64aa7cbc3f8a24053a => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:08.726983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0cd08-132e-4c87-8d76-2a3d09e177b4 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:08.741282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:08.782990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f15f8-fee7-49fc-b360-b30ed4e6e241 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:08.817362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e451e0-a2d8-4a41-aab3-d8e219f7cb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:08.817991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e451e0-a2d8-4a41-aab3-d8e219f7cb96 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:08.818269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e451e0-a2d8-4a41-aab3-d8e219f7cb96 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:08.818543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e451e0-a2d8-4a41-aab3-d8e219f7cb96 None admin] PATH_INFO: `/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:08.819639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e451e0-a2d8-4a41-aab3-d8e219f7cb96 None admin] RBAC: Authorizing `identity:get_group(group_id=dee1bc247ed841dfa4ca1d3d87f2ca9d)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:08.821468 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e451e0-a2d8-4a41-aab3-d8e219f7cb96 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:08.823054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 227/452] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:33:08 2022] GET /identity/v3/groups/dee1bc247ed841dfa4ca1d3d87f2ca9d => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:08.828737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e451e0-a2d8-4a41-aab3-d8e219f7cb96 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:08.831031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0cd08-132e-4c87-8d76-2a3d09e177b4 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:08.831410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:08.865825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125b2ba-2c75-4350-a075-b90581653c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:08.866446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125b2ba-2c75-4350-a075-b90581653c4d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:08.866807 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125b2ba-2c75-4350-a075-b90581653c4d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:08.867224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125b2ba-2c75-4350-a075-b90581653c4d None admin] PATH_INFO: `/v3/groups/8e4c593ada3d4c20a62fefe0322524f6` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:08.868352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125b2ba-2c75-4350-a075-b90581653c4d None admin] RBAC: Authorizing `identity:get_group(group_id=8e4c593ada3d4c20a62fefe0322524f6)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:08.869840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125b2ba-2c75-4350-a075-b90581653c4d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:08.871550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 226/453] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 8 18:33:08 2022] GET /identity/v3/groups/8e4c593ada3d4c20a62fefe0322524f6 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:08.877261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125b2ba-2c75-4350-a075-b90581653c4d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:08.929354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e451e0-a2d8-4a41-aab3-d8e219f7cb96 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:08.958616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:08.963334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da30cf-4998-49ea-ab16-ef30af1d6746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:08.963991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da30cf-4998-49ea-ab16-ef30af1d6746 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:08.964312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da30cf-4998-49ea-ab16-ef30af1d6746 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:08.964586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da30cf-4998-49ea-ab16-ef30af1d6746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:08.965338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da30cf-4998-49ea-ab16-ef30af1d6746 None admin] RBAC: Adding query filter params (group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d, role.id=6b3a5d54d6044e9b8d8270e7ff3145e8, scope.project.id=9ef8ff3249dd4e84ad8692c06a441377) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:08.965646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da30cf-4998-49ea-ab16-ef30af1d6746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:08.967103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da30cf-4998-49ea-ab16-ef30af1d6746 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:08.973278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 228/454] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 18:33:08 2022] GET /identity/v3/role_assignments?group.id=dee1bc247ed841dfa4ca1d3d87f2ca9d&scope.project.id=9ef8ff3249dd4e84ad8692c06a441377&role.id=6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:08.977999 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125b2ba-2c75-4350-a075-b90581653c4d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:09.015101 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc981a0-6a17-4b0f-999b-8f6f588498ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:09.015718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc981a0-6a17-4b0f-999b-8f6f588498ea None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:09.016000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc981a0-6a17-4b0f-999b-8f6f588498ea None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:09.016276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc981a0-6a17-4b0f-999b-8f6f588498ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:09.017019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc981a0-6a17-4b0f-999b-8f6f588498ea None admin] RBAC: Adding query filter params (group.id=8e4c593ada3d4c20a62fefe0322524f6, role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=1a0304ec85d64b64aa7cbc3f8a24053a) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:09.017360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc981a0-6a17-4b0f-999b-8f6f588498ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:09.018818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc981a0-6a17-4b0f-999b-8f6f588498ea None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:09.026096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 227/455] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 8 18:33:08 2022] GET /identity/v3/role_assignments?group.id=8e4c593ada3d4c20a62fefe0322524f6&scope.project.id=1a0304ec85d64b64aa7cbc3f8a24053a&role.id=5c777de40b8449329d714d8c2229115e => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:11.851645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae86ff4-74e3-4a3d-aecb-a641d025687e None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:11.852641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae86ff4-74e3-4a3d-aecb-a641d025687e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:11.853196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae86ff4-74e3-4a3d-aecb-a641d025687e None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:11.854815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 229/456] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:11.860999 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d4a2f-bfb4-4e29-8764-a78d072957c6 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:11.861933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d4a2f-bfb4-4e29-8764-a78d072957c6 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:11.862410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d4a2f-bfb4-4e29-8764-a78d072957c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:11.882481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-573d4a2f-bfb4-4e29-8764-a78d072957c6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:11.884148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-573d4a2f-bfb4-4e29-8764-a78d072957c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:11.914342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cb3d0-d24a-40a5-98ae-ad4ea9c6b119 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:11.914960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cb3d0-d24a-40a5-98ae-ad4ea9c6b119 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:11.915440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cb3d0-d24a-40a5-98ae-ad4ea9c6b119 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:11.917294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 230/457] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:11.922381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d5d81-2f9a-473d-8eb0-dd3cc17307b1 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:11.922925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d5d81-2f9a-473d-8eb0-dd3cc17307b1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:11.923391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d5d81-2f9a-473d-8eb0-dd3cc17307b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:11.948176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7f6d5d81-2f9a-473d-8eb0-dd3cc17307b1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:11.949744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7f6d5d81-2f9a-473d-8eb0-dd3cc17307b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:11.986796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 228/458] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:12.038302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:12.054368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 231/459] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:12.250795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4986e-43f9-499f-96b9-3bd8c4de8bec None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:12.251399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4986e-43f9-499f-96b9-3bd8c4de8bec None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:12.251885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4986e-43f9-499f-96b9-3bd8c4de8bec None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:12.253802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 229/460] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:12.259276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922e56b-6ac7-4f3b-970d-6c85d359c7a4 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:12.259780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922e56b-6ac7-4f3b-970d-6c85d359c7a4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:12.260247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922e56b-6ac7-4f3b-970d-6c85d359c7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:12.286035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8922e56b-6ac7-4f3b-970d-6c85d359c7a4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:12.287558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8922e56b-6ac7-4f3b-970d-6c85d359c7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:12.372593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 232/461] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:12.379059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4986e-43f9-499f-96b9-3bd8c4de8bec None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:12.448371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922e56b-6ac7-4f3b-970d-6c85d359c7a4 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:12.519783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:12.579254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4986e-43f9-499f-96b9-3bd8c4de8bec None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:12.655089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922e56b-6ac7-4f3b-970d-6c85d359c7a4 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:12.657741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a36b7-8146-44e9-9fd2-4a667aa2e0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:12.658374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a36b7-8146-44e9-9fd2-4a667aa2e0c1 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:12.658648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a36b7-8146-44e9-9fd2-4a667aa2e0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:12.658919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a36b7-8146-44e9-9fd2-4a667aa2e0c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:12.675118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a36b7-8146-44e9-9fd2-4a667aa2e0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:12.676897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a36b7-8146-44e9-9fd2-4a667aa2e0c1 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:12.689689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-832a36b7-8146-44e9-9fd2-4a667aa2e0c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:12.693646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:12.694527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:12.696383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:12.696383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:12.696383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:12.696383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 230/462] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:12.696607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a36b7-8146-44e9-9fd2-4a667aa2e0c1 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:12.703038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e03e2-eaa1-4b8b-adb7-93cf99c47d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:12.703688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e03e2-eaa1-4b8b-adb7-93cf99c47d21 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:12.703970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e03e2-eaa1-4b8b-adb7-93cf99c47d21 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:12.704718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e03e2-eaa1-4b8b-adb7-93cf99c47d21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:12.715263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e03e2-eaa1-4b8b-adb7-93cf99c47d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:12.716700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e03e2-eaa1-4b8b-adb7-93cf99c47d21 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-1b3e03e2-eaa1-4b8b-adb7-93cf99c47d21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:12.732626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:12.733638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:12.734572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:12.734572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:12.734572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:12.734572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 233/463] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:12.734898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5868-950d-401e-ac15-25702036d7d5 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:12.735198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5868-950d-401e-ac15-25702036d7d5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:12.735450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5868-950d-401e-ac15-25702036d7d5 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:12.736819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 234/464] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:12.741395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff5868-950d-401e-ac15-25702036d7d5 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:12.813999 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a36b7-8146-44e9-9fd2-4a667aa2e0c1 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:12.841408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:12.851821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41130-3d37-44b2-b8ae-5514758c2d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:12.852710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41130-3d37-44b2-b8ae-5514758c2d8b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:12.852993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41130-3d37-44b2-b8ae-5514758c2d8b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:12.853553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41130-3d37-44b2-b8ae-5514758c2d8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:12.863512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff5868-950d-401e-ac15-25702036d7d5 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:12.863972 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d41130-3d37-44b2-b8ae-5514758c2d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:12.865329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d41130-3d37-44b2-b8ae-5514758c2d8b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-b8d41130-3d37-44b2-b8ae-5514758c2d8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:12.880014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:12.881171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:12.882841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:12.882841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:12.882841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:12.889437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 231/465] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:12.889437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41130-3d37-44b2-b8ae-5514758c2d8b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:12.902951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a6d7f-a084-4a4e-9a8a-9921979a3cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:12.904155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a6d7f-a084-4a4e-9a8a-9921979a3cc7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:12.904584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a6d7f-a084-4a4e-9a8a-9921979a3cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:12.904871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a6d7f-a084-4a4e-9a8a-9921979a3cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:12.905604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a6d7f-a084-4a4e-9a8a-9921979a3cc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:12.906086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a6d7f-a084-4a4e-9a8a-9921979a3cc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:12.907850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a6d7f-a084-4a4e-9a8a-9921979a3cc7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:12.921764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 235/466] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:12.922986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc8260-ac79-40d5-88fb-6a06e9b75fce None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:12.923271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc8260-ac79-40d5-88fb-6a06e9b75fce None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:12.923543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc8260-ac79-40d5-88fb-6a06e9b75fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:12.945469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f4cc8260-ac79-40d5-88fb-6a06e9b75fce None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:12.946710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f4cc8260-ac79-40d5-88fb-6a06e9b75fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:12.981696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:13.001715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41130-3d37-44b2-b8ae-5514758c2d8b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:13.021694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 236/467] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:13.030609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc8260-ac79-40d5-88fb-6a06e9b75fce None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:13.038034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cf59e-8b67-4759-9f81-ce50af0d2586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:13.039203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cf59e-8b67-4759-9f81-ce50af0d2586 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:13.039476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cf59e-8b67-4759-9f81-ce50af0d2586 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:13.039754 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cf59e-8b67-4759-9f81-ce50af0d2586 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:13.040403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cf59e-8b67-4759-9f81-ce50af0d2586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:13.040710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cf59e-8b67-4759-9f81-ce50af0d2586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:13.042152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cf59e-8b67-4759-9f81-ce50af0d2586 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:13.064874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 232/468] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:13.064874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cf59e-8b67-4759-9f81-ce50af0d2586 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:13.121068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:13.151763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc8260-ac79-40d5-88fb-6a06e9b75fce None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:13.192986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f0327-4377-411f-ab91-0f79ecea23a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:13.193952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0327-4377-411f-ab91-0f79ecea23a4 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:13.194375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0327-4377-411f-ab91-0f79ecea23a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:13.194961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0327-4377-411f-ab91-0f79ecea23a4 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:13.196090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f0327-4377-411f-ab91-0f79ecea23a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:13.196826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f0327-4377-411f-ab91-0f79ecea23a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:13.198542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f0327-4377-411f-ab91-0f79ecea23a4 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:13.201898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cf59e-8b67-4759-9f81-ce50af0d2586 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:13.213824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 237/469] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:13.214817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f0327-4377-411f-ab91-0f79ecea23a4 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:13.243965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc9a19-9ed3-4cae-af43-aaaa58d683a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:13.244721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9a19-9ed3-4cae-af43-aaaa58d683a9 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:13.245130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9a19-9ed3-4cae-af43-aaaa58d683a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:13.245509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9a19-9ed3-4cae-af43-aaaa58d683a9 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:13.246281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc9a19-9ed3-4cae-af43-aaaa58d683a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:13.247837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc9a19-9ed3-4cae-af43-aaaa58d683a9 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:13.251013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:13.331310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f0327-4377-411f-ab91-0f79ecea23a4 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:13.341930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 233/470] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:33:13 2022] POST /identity/v3/users => generated 246 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:33:13.354144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc9a19-9ed3-4cae-af43-aaaa58d683a9 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:13.372297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58247d-e430-4d56-8447-82e83c4c9ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:13.372930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58247d-e430-4d56-8447-82e83c4c9ef9 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:13.373247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58247d-e430-4d56-8447-82e83c4c9ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:13.373518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58247d-e430-4d56-8447-82e83c4c9ef9 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:13.374188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58247d-e430-4d56-8447-82e83c4c9ef9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:13.376097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58247d-e430-4d56-8447-82e83c4c9ef9 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:13.455998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 238/471] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:33:13 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:33:13.456771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58247d-e430-4d56-8447-82e83c4c9ef9 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:13.487349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:13.495586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc9a19-9ed3-4cae-af43-aaaa58d683a9 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:13.557478 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e2e24-b7ec-4988-9618-b6863867e07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:13.558114 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e2e24-b7ec-4988-9618-b6863867e07d None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:13.558561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e2e24-b7ec-4988-9618-b6863867e07d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:13.558952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e2e24-b7ec-4988-9618-b6863867e07d None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:13.559629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e2e24-b7ec-4988-9618-b6863867e07d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:13.562027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e2e24-b7ec-4988-9618-b6863867e07d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:13.596962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:13.615103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58247d-e430-4d56-8447-82e83c4c9ef9 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:13.653231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3175d1-7c14-43ca-b3ca-3732cb79d27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:13.653876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3175d1-7c14-43ca-b3ca-3732cb79d27b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:13.654156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3175d1-7c14-43ca-b3ca-3732cb79d27b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:13.654431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3175d1-7c14-43ca-b3ca-3732cb79d27b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:13.662856 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 234/472] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:33:13 2022] POST /identity/v3/users => generated 248 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:33:13.665719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cdf8d-58ca-4494-a170-42f4aa6c2e85 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:13.666110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3175d1-7c14-43ca-b3ca-3732cb79d27b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:13.666428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cdf8d-58ca-4494-a170-42f4aa6c2e85 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:13.666888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cdf8d-58ca-4494-a170-42f4aa6c2e85 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:13.667561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3175d1-7c14-43ca-b3ca-3732cb79d27b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:13.669172 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 235/473] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:13.674780 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a090-dd10-487f-96dd-bee5b91ae27d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:13.675971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a090-dd10-487f-96dd-bee5b91ae27d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:13.676574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a090-dd10-487f-96dd-bee5b91ae27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-9d3175d1-7c14-43ca-b3ca-3732cb79d27b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:13.684796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:13.685865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:13.686792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:13.686792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:13.686792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:13.686792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 239/474] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:13.694578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3175d1-7c14-43ca-b3ca-3732cb79d27b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:13.707185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-86f2a090-dd10-487f-96dd-bee5b91ae27d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:13.709632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-86f2a090-dd10-487f-96dd-bee5b91ae27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:13.795235 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 236/475] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:13.820723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3175d1-7c14-43ca-b3ca-3732cb79d27b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:13.860240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbbcea-87a4-4d27-a2e5-cf93c1b59c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:13.860789 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbbcea-87a4-4d27-a2e5-cf93c1b59c07 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:13.861086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbbcea-87a4-4d27-a2e5-cf93c1b59c07 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:13.861365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbbcea-87a4-4d27-a2e5-cf93c1b59c07 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:13.862027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbbcea-87a4-4d27-a2e5-cf93c1b59c07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:13.862337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbbcea-87a4-4d27-a2e5-cf93c1b59c07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:13.863739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbbcea-87a4-4d27-a2e5-cf93c1b59c07 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:13.867098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:13.877729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 240/476] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:13.887228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2a090-dd10-487f-96dd-bee5b91ae27d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:13.993515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:14.016548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2a090-dd10-487f-96dd-bee5b91ae27d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:14.061204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a45bc-e55a-439f-8217-cce239306e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:14.062128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a45bc-e55a-439f-8217-cce239306e1f None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:14.062597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a45bc-e55a-439f-8217-cce239306e1f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:14.063052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a45bc-e55a-439f-8217-cce239306e1f None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:14.063884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a45bc-e55a-439f-8217-cce239306e1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:14.065584 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a45bc-e55a-439f-8217-cce239306e1f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:14.093967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbbcea-87a4-4d27-a2e5-cf93c1b59c07 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:14.141535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 237/477] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:33:13 2022] POST /identity/v3/users => generated 249 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:33:14.247837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbbcea-87a4-4d27-a2e5-cf93c1b59c07 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:14.305198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06b5b2-2df8-4fe4-835e-639b4966bd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:14.305821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06b5b2-2df8-4fe4-835e-639b4966bd47 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:14.306099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06b5b2-2df8-4fe4-835e-639b4966bd47 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:14.306368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06b5b2-2df8-4fe4-835e-639b4966bd47 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:14.307059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06b5b2-2df8-4fe4-835e-639b4966bd47 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:14.308447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06b5b2-2df8-4fe4-835e-639b4966bd47 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:14.349339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.sql.core [None req-5b06b5b2-2df8-4fe4-835e-639b4966bd47 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 18:33:14.349339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 18:33:14.349339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [parameters: {'id': '68bddb6327214d71b4ce5e8828f7e492', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 18:33:14.349339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88173) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-5b06b5b2-2df8-4fe4-835e-639b4966bd47 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application conn.query(q) Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application result.read() Sep 08 18:33:14.370699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return ref.to_dict() Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 18:33:14.371670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application next(self.gen) Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application yield resource Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application next(self.gen) Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise self.value Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application session.commit() Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 18:33:14.372605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self.session.flush() Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application compat.raise_( Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 18:33:14.373512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 18:33:14.374433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application conn.query(q) Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application result.read() Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 18:33:14.375252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [parameters: {'id': '68bddb6327214d71b4ce5e8828f7e492', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:14.376038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 18:33:14.380897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:14.383658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 241/478] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 8 18:33:14 2022] POST /identity/v3/roles => generated 144 bytes in 279 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:14.383799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a45bc-e55a-439f-8217-cce239306e1f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:14.520459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a45bc-e55a-439f-8217-cce239306e1f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:14.560067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdf6af-5ef3-4448-b306-b74165b65d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:14.561127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdf6af-5ef3-4448-b306-b74165b65d36 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:14.561641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdf6af-5ef3-4448-b306-b74165b65d36 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:14.562199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdf6af-5ef3-4448-b306-b74165b65d36 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:14.565542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:14.579761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdf6af-5ef3-4448-b306-b74165b65d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-dbbdf6af-5ef3-4448-b306-b74165b65d36 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:14.585937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:14.587533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 18:33:14.588344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:14.590317 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 238/479] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:33:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:14.596356 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06b5b2-2df8-4fe4-835e-639b4966bd47 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:14.666202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:14.711625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06b5b2-2df8-4fe4-835e-639b4966bd47 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:14.749303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534c417-b8f5-4416-8dfe-572b9a254c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:14.749974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534c417-b8f5-4416-8dfe-572b9a254c2c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:14.750259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534c417-b8f5-4416-8dfe-572b9a254c2c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:14.750536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534c417-b8f5-4416-8dfe-572b9a254c2c None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:14.751215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534c417-b8f5-4416-8dfe-572b9a254c2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:14.751520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534c417-b8f5-4416-8dfe-572b9a254c2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:14.752878 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534c417-b8f5-4416-8dfe-572b9a254c2c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:14.762688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 242/480] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:15.994985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6d579-1222-4ff6-885e-f315894e2dac None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:15.995566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6d579-1222-4ff6-885e-f315894e2dac None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:15.996130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6d579-1222-4ff6-885e-f315894e2dac None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:16.006813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 239/481] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:16.009147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73069c-fb69-47f0-bc6f-f48a64300f5d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:16.009583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73069c-fb69-47f0-bc6f-f48a64300f5d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:16.009675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73069c-fb69-47f0-bc6f-f48a64300f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:16.036461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7e73069c-fb69-47f0-bc6f-f48a64300f5d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:16.037718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7e73069c-fb69-47f0-bc6f-f48a64300f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:16.128013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 243/482] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:16.315477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116a89f-ac93-4db2-ad2e-9df1ebb6e7f4 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:16.315776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116a89f-ac93-4db2-ad2e-9df1ebb6e7f4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:16.316041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116a89f-ac93-4db2-ad2e-9df1ebb6e7f4 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:16.317699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 240/483] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:16.325342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0bd90-24c4-4d5c-8c2c-ca6030b52be1 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:16.325685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0bd90-24c4-4d5c-8c2c-ca6030b52be1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:16.325919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0bd90-24c4-4d5c-8c2c-ca6030b52be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:16.355329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-32a0bd90-24c4-4d5c-8c2c-ca6030b52be1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:16.358089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-32a0bd90-24c4-4d5c-8c2c-ca6030b52be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:16.450967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:16.483153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 244/484] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:16.494936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38699697-231d-4f6a-a82c-2d081c41f6ed None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:16.495573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38699697-231d-4f6a-a82c-2d081c41f6ed None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:16.496168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38699697-231d-4f6a-a82c-2d081c41f6ed None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:16.499357 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 241/485] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:16.506807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef3888-1581-4a92-a33f-0e13f2f33785 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:16.507694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef3888-1581-4a92-a33f-0e13f2f33785 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:16.507982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef3888-1581-4a92-a33f-0e13f2f33785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:16.540600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-02ef3888-1581-4a92-a33f-0e13f2f33785 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:16.541834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-02ef3888-1581-4a92-a33f-0e13f2f33785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:16.557789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38699697-231d-4f6a-a82c-2d081c41f6ed None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:16.626393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 245/486] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:16.756897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38699697-231d-4f6a-a82c-2d081c41f6ed None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:16.809817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a583742-3895-49a0-8738-6e09b504c4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:16.810475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a583742-3895-49a0-8738-6e09b504c4c4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:16.810809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a583742-3895-49a0-8738-6e09b504c4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:16.811080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a583742-3895-49a0-8738-6e09b504c4c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:16.828734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a583742-3895-49a0-8738-6e09b504c4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-3a583742-3895-49a0-8738-6e09b504c4c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:16.834109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:16.836499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:16.837681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 242/487] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:16.843594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef3888-1581-4a92-a33f-0e13f2f33785 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:16.971000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a583742-3895-49a0-8738-6e09b504c4c4 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:17.010385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef3888-1581-4a92-a33f-0e13f2f33785 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:17.028912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:17.054869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686212d2-9c07-4f94-99bb-f6051d2de46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:17.055653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686212d2-9c07-4f94-99bb-f6051d2de46c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.055929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686212d2-9c07-4f94-99bb-f6051d2de46c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.056170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686212d2-9c07-4f94-99bb-f6051d2de46c None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.056849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686212d2-9c07-4f94-99bb-f6051d2de46c None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:17.057184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686212d2-9c07-4f94-99bb-f6051d2de46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:17.058553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686212d2-9c07-4f94-99bb-f6051d2de46c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:17.068585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 246/488] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:17.073033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686212d2-9c07-4f94-99bb-f6051d2de46c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:17.105339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:17.166351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a583742-3895-49a0-8738-6e09b504c4c4 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:17.215463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686212d2-9c07-4f94-99bb-f6051d2de46c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:17.231177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3920643-580d-4212-ab07-c1d7df9e88f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:17.231744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3920643-580d-4212-ab07-c1d7df9e88f5 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.232023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3920643-580d-4212-ab07-c1d7df9e88f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.232300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3920643-580d-4212-ab07-c1d7df9e88f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.245665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3920643-580d-4212-ab07-c1d7df9e88f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-c3920643-580d-4212-ab07-c1d7df9e88f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:17.255554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:17.256533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:17.257508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:17.258112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f410759-4eba-4c9b-89c5-f61d5d8e41c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:17.258112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f410759-4eba-4c9b-89c5-f61d5d8e41c0 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.258265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 243/489] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:17.258339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f410759-4eba-4c9b-89c5-f61d5d8e41c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.258927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f410759-4eba-4c9b-89c5-f61d5d8e41c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.265069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3920643-580d-4212-ab07-c1d7df9e88f5 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:17.279407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f410759-4eba-4c9b-89c5-f61d5d8e41c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-6f410759-4eba-4c9b-89c5-f61d5d8e41c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:17.285871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:17.287981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 247/490] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:17.290217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f410759-4eba-4c9b-89c5-f61d5d8e41c0 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:17.331401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:17.399965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3920643-580d-4212-ab07-c1d7df9e88f5 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:17.408548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f410759-4eba-4c9b-89c5-f61d5d8e41c0 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:17.419138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:17.441947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a599c69-681e-42da-808b-b102f1c06a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:17.442563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a599c69-681e-42da-808b-b102f1c06a11 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.442846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a599c69-681e-42da-808b-b102f1c06a11 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.443124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a599c69-681e-42da-808b-b102f1c06a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.445073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba65f6-4e8c-4c8c-96f5-eddfdd9833f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:17.445687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba65f6-4e8c-4c8c-96f5-eddfdd9833f6 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.445959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba65f6-4e8c-4c8c-96f5-eddfdd9833f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.446240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba65f6-4e8c-4c8c-96f5-eddfdd9833f6 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.446899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba65f6-4e8c-4c8c-96f5-eddfdd9833f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:17.447203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba65f6-4e8c-4c8c-96f5-eddfdd9833f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:17.448566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba65f6-4e8c-4c8c-96f5-eddfdd9833f6 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:17.454581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a599c69-681e-42da-808b-b102f1c06a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:17.455714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a599c69-681e-42da-808b-b102f1c06a11 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:17.462299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 248/491] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:17.462299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba65f6-4e8c-4c8c-96f5-eddfdd9833f6 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-7a599c69-681e-42da-808b-b102f1c06a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:17.476790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:17.477879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:17.478782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:17.478782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:17.478782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:17.478782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 244/492] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:17.479496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c231fc3-2957-4145-a2f6-d1c9c1679fc4 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.479781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c231fc3-2957-4145-a2f6-d1c9c1679fc4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.480054 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c231fc3-2957-4145-a2f6-d1c9c1679fc4 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.481642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 245/493] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:17.482294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c231fc3-2957-4145-a2f6-d1c9c1679fc4 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:17.572363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:17.578205 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba65f6-4e8c-4c8c-96f5-eddfdd9833f6 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:17.596504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c231fc3-2957-4145-a2f6-d1c9c1679fc4 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:17.618064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670f801-7047-41f0-a89a-01c5262b3bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:17.618679 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670f801-7047-41f0-a89a-01c5262b3bd3 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.618961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670f801-7047-41f0-a89a-01c5262b3bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.619242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670f801-7047-41f0-a89a-01c5262b3bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.619895 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670f801-7047-41f0-a89a-01c5262b3bd3 None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:17.620196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670f801-7047-41f0-a89a-01c5262b3bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:17.621587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670f801-7047-41f0-a89a-01c5262b3bd3 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:17.630978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 249/494] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:17.631616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670f801-7047-41f0-a89a-01c5262b3bd3 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:17.636261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be14438-fcdb-4cb0-b625-02c75f7e0f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:17.637014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be14438-fcdb-4cb0-b625-02c75f7e0f96 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.637320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be14438-fcdb-4cb0-b625-02c75f7e0f96 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.637598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be14438-fcdb-4cb0-b625-02c75f7e0f96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.652680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be14438-fcdb-4cb0-b625-02c75f7e0f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:17.654179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be14438-fcdb-4cb0-b625-02c75f7e0f96 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-2be14438-fcdb-4cb0-b625-02c75f7e0f96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:17.668407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:17.669330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:17.670212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:17.670212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:17.670212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:17.674158 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 246/495] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:17.675160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae162f55-c60f-4be8-88a1-e2c5b6e73ae9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.675448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae162f55-c60f-4be8-88a1-e2c5b6e73ae9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.675724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae162f55-c60f-4be8-88a1-e2c5b6e73ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.694592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ae162f55-c60f-4be8-88a1-e2c5b6e73ae9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:17.696414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ae162f55-c60f-4be8-88a1-e2c5b6e73ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:17.712337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:17.743731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670f801-7047-41f0-a89a-01c5262b3bd3 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:17.770688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 247/496] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:17.771613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae162f55-c60f-4be8-88a1-e2c5b6e73ae9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:17.802125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51543c2b-84e8-4970-babc-555e4ef7ecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:17.802744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51543c2b-84e8-4970-babc-555e4ef7ecce None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.803027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51543c2b-84e8-4970-babc-555e4ef7ecce None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.803307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51543c2b-84e8-4970-babc-555e4ef7ecce None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.804555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51543c2b-84e8-4970-babc-555e4ef7ecce None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:17.804980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51543c2b-84e8-4970-babc-555e4ef7ecce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:17.806402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51543c2b-84e8-4970-babc-555e4ef7ecce None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:17.821293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 250/497] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:17.826740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51543c2b-84e8-4970-babc-555e4ef7ecce None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:17.848446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:17.912517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae162f55-c60f-4be8-88a1-e2c5b6e73ae9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:17.950480 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe239-382c-4c3f-97e0-220a6b269628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:17.951408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe239-382c-4c3f-97e0-220a6b269628 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:17.951694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe239-382c-4c3f-97e0-220a6b269628 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:17.951972 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe239-382c-4c3f-97e0-220a6b269628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:17.957336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51543c2b-84e8-4970-babc-555e4ef7ecce None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:17.964667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfe239-382c-4c3f-97e0-220a6b269628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:17.966187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfe239-382c-4c3f-97e0-220a6b269628 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-ebbfe239-382c-4c3f-97e0-220a6b269628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:17.983399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:17.984294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:17.985155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:17.985155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:17.985155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:17.986230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 248/498] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:17.987013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe239-382c-4c3f-97e0-220a6b269628 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:17.991852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:18.005482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b89e74-b93f-43e6-84ee-8c8d44f67084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.005482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b89e74-b93f-43e6-84ee-8c8d44f67084 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.005482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b89e74-b93f-43e6-84ee-8c8d44f67084 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.005482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b89e74-b93f-43e6-84ee-8c8d44f67084 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.005833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b89e74-b93f-43e6-84ee-8c8d44f67084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:18.005833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b89e74-b93f-43e6-84ee-8c8d44f67084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:18.005833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b89e74-b93f-43e6-84ee-8c8d44f67084 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:18.015052 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 251/499] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:18.016357 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98e1eb-482b-4d60-b837-a8001209c243 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.016722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98e1eb-482b-4d60-b837-a8001209c243 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.017263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98e1eb-482b-4d60-b837-a8001209c243 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.022853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 252/500] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:18.022853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98e1eb-482b-4d60-b837-a8001209c243 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:18.097618 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe239-382c-4c3f-97e0-220a6b269628 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:18.106730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:18.129988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98e1eb-482b-4d60-b837-a8001209c243 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:18.132894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37051c17-a009-48e7-9bc8-6fe5c68449a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.133551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37051c17-a009-48e7-9bc8-6fe5c68449a7 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.133829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37051c17-a009-48e7-9bc8-6fe5c68449a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.134102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37051c17-a009-48e7-9bc8-6fe5c68449a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.141627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37051c17-a009-48e7-9bc8-6fe5c68449a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:18.143129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37051c17-a009-48e7-9bc8-6fe5c68449a7 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-37051c17-a009-48e7-9bc8-6fe5c68449a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:18.155830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:18.156797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 249/501] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 18:33:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:18.157739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37051c17-a009-48e7-9bc8-6fe5c68449a7 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:18.167918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27874fd6-82ac-4916-addf-d18ae714ec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.168521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27874fd6-82ac-4916-addf-d18ae714ec1f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.168796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27874fd6-82ac-4916-addf-d18ae714ec1f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.169087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27874fd6-82ac-4916-addf-d18ae714ec1f None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.169721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27874fd6-82ac-4916-addf-d18ae714ec1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:18.170018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27874fd6-82ac-4916-addf-d18ae714ec1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:18.171409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27874fd6-82ac-4916-addf-d18ae714ec1f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:18.183531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 253/502] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:18.184706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5836a9-70d4-4c45-975d-a0bdb17c179d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.184993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5836a9-70d4-4c45-975d-a0bdb17c179d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.185302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5836a9-70d4-4c45-975d-a0bdb17c179d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.203167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-ca5836a9-70d4-4c45-975d-a0bdb17c179d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:18.204347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ca5836a9-70d4-4c45-975d-a0bdb17c179d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:18.222510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:18.263470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37051c17-a009-48e7-9bc8-6fe5c68449a7 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:18.270318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 254/503] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:18.271144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5836a9-70d4-4c45-975d-a0bdb17c179d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:18.298011 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746057b-f928-44f9-932a-c364ee119a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.298640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746057b-f928-44f9-932a-c364ee119a41 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.298922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746057b-f928-44f9-932a-c364ee119a41 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.299198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746057b-f928-44f9-932a-c364ee119a41 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.308236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746057b-f928-44f9-932a-c364ee119a41 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:18.309747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746057b-f928-44f9-932a-c364ee119a41 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-2746057b-f928-44f9-932a-c364ee119a41 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:18.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:18.322335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 250/504] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:18.323239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746057b-f928-44f9-932a-c364ee119a41 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:18.353537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:18.392096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5836a9-70d4-4c45-975d-a0bdb17c179d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:18.428472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057d3c7-d1d3-4d71-88b8-6f74fbe83c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.429972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057d3c7-d1d3-4d71-88b8-6f74fbe83c13 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.430432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057d3c7-d1d3-4d71-88b8-6f74fbe83c13 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.430895 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057d3c7-d1d3-4d71-88b8-6f74fbe83c13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.432678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746057b-f928-44f9-932a-c364ee119a41 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:18.443929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057d3c7-d1d3-4d71-88b8-6f74fbe83c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-1057d3c7-d1d3-4d71-88b8-6f74fbe83c13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:18.449256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:18.455088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 255/505] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:18.455900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057d3c7-d1d3-4d71-88b8-6f74fbe83c13 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:18.466418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:18.478503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb3663-ea1d-4ecb-875e-1fb81860dff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.482955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb3663-ea1d-4ecb-875e-1fb81860dff8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.482955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb3663-ea1d-4ecb-875e-1fb81860dff8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.482955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb3663-ea1d-4ecb-875e-1fb81860dff8 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.482955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb3663-ea1d-4ecb-875e-1fb81860dff8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:18.482955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb3663-ea1d-4ecb-875e-1fb81860dff8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:18.482955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb3663-ea1d-4ecb-875e-1fb81860dff8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:18.527240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 251/506] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:18.527240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb3663-ea1d-4ecb-875e-1fb81860dff8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:18.574972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057d3c7-d1d3-4d71-88b8-6f74fbe83c13 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:18.605004 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:18.612797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579478d1-7e6e-43dc-9e9b-25810282eb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.613469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579478d1-7e6e-43dc-9e9b-25810282eb8e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.613750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579478d1-7e6e-43dc-9e9b-25810282eb8e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.614541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579478d1-7e6e-43dc-9e9b-25810282eb8e None admin] PATH_INFO: `/v3/users/glance` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.621767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579478d1-7e6e-43dc-9e9b-25810282eb8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:18.623281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579478d1-7e6e-43dc-9e9b-25810282eb8e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:18.626070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb3663-ea1d-4ecb-875e-1fb81860dff8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-579478d1-7e6e-43dc-9e9b-25810282eb8e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:18.637439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:18.638404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:18.639291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:18.639291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:18.639291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:18.639291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:18.639291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:18.639291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:18.639291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:18.639291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:18.639291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 18:33:18.639291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:18.643231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 256/507] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:18.643944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579478d1-7e6e-43dc-9e9b-25810282eb8e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:18.660689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d4c01-5e43-4b35-aeb7-2e27b55d8d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.661334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d4c01-5e43-4b35-aeb7-2e27b55d8d8b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.661622 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d4c01-5e43-4b35-aeb7-2e27b55d8d8b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.661901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d4c01-5e43-4b35-aeb7-2e27b55d8d8b None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.662628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d4c01-5e43-4b35-aeb7-2e27b55d8d8b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:18.662935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d4c01-5e43-4b35-aeb7-2e27b55d8d8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:18.664515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d4c01-5e43-4b35-aeb7-2e27b55d8d8b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:18.682482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 252/508] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:18.683294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d4c01-5e43-4b35-aeb7-2e27b55d8d8b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:18.722429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:18.745932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579478d1-7e6e-43dc-9e9b-25810282eb8e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:18.779177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0463202-ae1c-4ded-8c47-fa2591077bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.779748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0463202-ae1c-4ded-8c47-fa2591077bf8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.780028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0463202-ae1c-4ded-8c47-fa2591077bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.780305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0463202-ae1c-4ded-8c47-fa2591077bf8 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.780970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0463202-ae1c-4ded-8c47-fa2591077bf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:18.781304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0463202-ae1c-4ded-8c47-fa2591077bf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:18.781394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d4c01-5e43-4b35-aeb7-2e27b55d8d8b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:18.782704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0463202-ae1c-4ded-8c47-fa2591077bf8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:18.792817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 257/509] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:18.793705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0463202-ae1c-4ded-8c47-fa2591077bf8 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:18.817522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ef753-5fef-4cf6-8e75-7426ecb43e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.818131 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ef753-5fef-4cf6-8e75-7426ecb43e13 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.818411 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ef753-5fef-4cf6-8e75-7426ecb43e13 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.818689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ef753-5fef-4cf6-8e75-7426ecb43e13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.843491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:18.885389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ef753-5fef-4cf6-8e75-7426ecb43e13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:18.886838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ef753-5fef-4cf6-8e75-7426ecb43e13 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:18.893185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0463202-ae1c-4ded-8c47-fa2591077bf8 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:18.924812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:18.928150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe11d26-2a82-4178-a2ec-b6b335819eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:18.931827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe11d26-2a82-4178-a2ec-b6b335819eaf None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:18.931827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe11d26-2a82-4178-a2ec-b6b335819eaf None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:18.931827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe11d26-2a82-4178-a2ec-b6b335819eaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:18.941263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe11d26-2a82-4178-a2ec-b6b335819eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:18.942692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe11d26-2a82-4178-a2ec-b6b335819eaf None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-afe11d26-2a82-4178-a2ec-b6b335819eaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:18.956579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:18.957721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:18.958564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:18.958564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:18.958564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:18.958564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 258/510] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:18.958564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe11d26-2a82-4178-a2ec-b6b335819eaf None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:18.988457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 253/511] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:18.989330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ef753-5fef-4cf6-8e75-7426ecb43e13 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.060736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe11d26-2a82-4178-a2ec-b6b335819eaf None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:19.062289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:19.090213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ef753-5fef-4cf6-8e75-7426ecb43e13 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:19.096133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08fb2bc-fdc8-4ba9-ad07-d286eb5df382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:19.096734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fb2bc-fdc8-4ba9-ad07-d286eb5df382 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:19.097009 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fb2bc-fdc8-4ba9-ad07-d286eb5df382 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:19.097319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08fb2bc-fdc8-4ba9-ad07-d286eb5df382 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:19.098036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08fb2bc-fdc8-4ba9-ad07-d286eb5df382 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:19.098340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08fb2bc-fdc8-4ba9-ad07-d286eb5df382 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:19.099764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08fb2bc-fdc8-4ba9-ad07-d286eb5df382 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:19.126591 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcc21b-23c6-468e-bb89-4a98c6e4aefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:19.127202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcc21b-23c6-468e-bb89-4a98c6e4aefb None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:19.127484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcc21b-23c6-468e-bb89-4a98c6e4aefb None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:19.127762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcc21b-23c6-468e-bb89-4a98c6e4aefb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:19.136898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:19.138753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 259/512] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:19.139903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08fb2bc-fdc8-4ba9-ad07-d286eb5df382 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.197355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcc21b-23c6-468e-bb89-4a98c6e4aefb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:19.198843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcc21b-23c6-468e-bb89-4a98c6e4aefb None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:19.239009 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08fb2bc-fdc8-4ba9-ad07-d286eb5df382 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:19.273467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a10585-b414-406d-89bb-55a321be9e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:19.274091 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a10585-b414-406d-89bb-55a321be9e59 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:19.274368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a10585-b414-406d-89bb-55a321be9e59 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:19.274643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a10585-b414-406d-89bb-55a321be9e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:19.284673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a10585-b414-406d-89bb-55a321be9e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:19.286139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a10585-b414-406d-89bb-55a321be9e59 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:19.295689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 254/513] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:19.296520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcc21b-23c6-468e-bb89-4a98c6e4aefb None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-b3a10585-b414-406d-89bb-55a321be9e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:19.300491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:19.301492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:19.302347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:19.302347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:19.302347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:19.302347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 260/514] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:19.302347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a10585-b414-406d-89bb-55a321be9e59 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.318936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:19.339913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:19.400847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcc21b-23c6-468e-bb89-4a98c6e4aefb None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:19.406807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a10585-b414-406d-89bb-55a321be9e59 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:19.434504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c18447-fbc8-49fd-aaa3-36e726102c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:19.435102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c18447-fbc8-49fd-aaa3-36e726102c35 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:19.435376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c18447-fbc8-49fd-aaa3-36e726102c35 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:19.435647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c18447-fbc8-49fd-aaa3-36e726102c35 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:19.436281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c18447-fbc8-49fd-aaa3-36e726102c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:19.436580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c18447-fbc8-49fd-aaa3-36e726102c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:19.437983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c18447-fbc8-49fd-aaa3-36e726102c35 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:19.441372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9ede-a40e-47fd-a666-eedab046d169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:19.441997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9ede-a40e-47fd-a666-eedab046d169 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:19.442275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9ede-a40e-47fd-a666-eedab046d169 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:19.442548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9ede-a40e-47fd-a666-eedab046d169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:19.449598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 255/515] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:19.450386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c18447-fbc8-49fd-aaa3-36e726102c35 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.454405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9ede-a40e-47fd-a666-eedab046d169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:19.455873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9ede-a40e-47fd-a666-eedab046d169 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-14ba9ede-a40e-47fd-a666-eedab046d169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:19.471195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:19.472238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:19.478552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:19.478552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:19.478552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:19.478552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 261/516] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:19.478552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9ede-a40e-47fd-a666-eedab046d169 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.534877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:19.548844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:19.555364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c18447-fbc8-49fd-aaa3-36e726102c35 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:19.577070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9ede-a40e-47fd-a666-eedab046d169 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:19.589904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e344df8-2daf-443e-89dc-45baadc9305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:19.590720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e344df8-2daf-443e-89dc-45baadc9305d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:19.590999 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e344df8-2daf-443e-89dc-45baadc9305d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:19.591269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e344df8-2daf-443e-89dc-45baadc9305d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:19.613881 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d05b31-a62b-4f4b-afd8-8ec6e2ad059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:19.614514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05b31-a62b-4f4b-afd8-8ec6e2ad059f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:19.614793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05b31-a62b-4f4b-afd8-8ec6e2ad059f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:19.615066 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05b31-a62b-4f4b-afd8-8ec6e2ad059f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:19.626117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d05b31-a62b-4f4b-afd8-8ec6e2ad059f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:19.627559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d05b31-a62b-4f4b-afd8-8ec6e2ad059f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-58d05b31-a62b-4f4b-afd8-8ec6e2ad059f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:19.641446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:19.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:19.643231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:19.643231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:19.643231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:19.643231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 262/517] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:19.643444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d05b31-a62b-4f4b-afd8-8ec6e2ad059f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.662344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e344df8-2daf-443e-89dc-45baadc9305d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:19.664345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e344df8-2daf-443e-89dc-45baadc9305d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:19.745310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d05b31-a62b-4f4b-afd8-8ec6e2ad059f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:19.756529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:19.761846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 256/518] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:19.762013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:19.762894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e344df8-2daf-443e-89dc-45baadc9305d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.783540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136ad84-054f-45d7-8a35-d80e42b8df87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:19.784143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136ad84-054f-45d7-8a35-d80e42b8df87 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:19.784416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136ad84-054f-45d7-8a35-d80e42b8df87 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:19.784685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136ad84-054f-45d7-8a35-d80e42b8df87 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:19.785360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136ad84-054f-45d7-8a35-d80e42b8df87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:19.785717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136ad84-054f-45d7-8a35-d80e42b8df87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:19.787084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136ad84-054f-45d7-8a35-d80e42b8df87 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:19.798909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 263/519] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:19.799699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136ad84-054f-45d7-8a35-d80e42b8df87 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.865782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e344df8-2daf-443e-89dc-45baadc9305d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:19.899644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceafee-6334-48f9-854b-a54c02e4fab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:19.900245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceafee-6334-48f9-854b-a54c02e4fab7 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:19.900528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceafee-6334-48f9-854b-a54c02e4fab7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:19.900801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceafee-6334-48f9-854b-a54c02e4fab7 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:19.901474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceafee-6334-48f9-854b-a54c02e4fab7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:19.902902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceafee-6334-48f9-854b-a54c02e4fab7 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:19.905096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136ad84-054f-45d7-8a35-d80e42b8df87 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:19.938704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6ef8-7394-430a-9e66-4ab390d67b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:19.939297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6ef8-7394-430a-9e66-4ab390d67b6f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:19.939575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6ef8-7394-430a-9e66-4ab390d67b6f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:19.939844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6ef8-7394-430a-9e66-4ab390d67b6f None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:19.940475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de6ef8-7394-430a-9e66-4ab390d67b6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:19.940775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de6ef8-7394-430a-9e66-4ab390d67b6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:19.942254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de6ef8-7394-430a-9e66-4ab390d67b6f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:19.953820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 264/520] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:19.954593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6ef8-7394-430a-9e66-4ab390d67b6f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.981249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 257/521] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:33:19 2022] POST /identity/v3/users => generated 247 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:33:19.982115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceafee-6334-48f9-854b-a54c02e4fab7 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:19.990085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:20.055164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:20.066288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6ef8-7394-430a-9e66-4ab390d67b6f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:20.104578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d7283-fa15-4c15-b689-2c7b54ff5fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:20.105531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d7283-fa15-4c15-b689-2c7b54ff5fee None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:20.105962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d7283-fa15-4c15-b689-2c7b54ff5fee None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:20.106232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d7283-fa15-4c15-b689-2c7b54ff5fee None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:20.106931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d7283-fa15-4c15-b689-2c7b54ff5fee None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:20.107233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d7283-fa15-4c15-b689-2c7b54ff5fee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:20.108485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceafee-6334-48f9-854b-a54c02e4fab7 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:20.108671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d7283-fa15-4c15-b689-2c7b54ff5fee None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:20.124866 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 265/522] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:20.126166 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d7283-fa15-4c15-b689-2c7b54ff5fee None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:20.151970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d14d6b-9efc-4227-bc1d-720c55a279fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:20.153076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d14d6b-9efc-4227-bc1d-720c55a279fb None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:20.153076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d14d6b-9efc-4227-bc1d-720c55a279fb None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:20.153352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d14d6b-9efc-4227-bc1d-720c55a279fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:20.166477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d14d6b-9efc-4227-bc1d-720c55a279fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:20.167934 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d14d6b-9efc-4227-bc1d-720c55a279fb None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-86d14d6b-9efc-4227-bc1d-720c55a279fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:20.184125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:20.185075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:20.185959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:20.185959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:20.185959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:20.188151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 258/523] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:20.189171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d14d6b-9efc-4227-bc1d-720c55a279fb None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:20.242547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:20.249673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d7283-fa15-4c15-b689-2c7b54ff5fee None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:20.287180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05bcce-dda5-4201-96e7-2804674a8ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:20.287817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05bcce-dda5-4201-96e7-2804674a8ed5 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:20.288100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05bcce-dda5-4201-96e7-2804674a8ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:20.288928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05bcce-dda5-4201-96e7-2804674a8ed5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:20.296273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d14d6b-9efc-4227-bc1d-720c55a279fb None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:20.296417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05bcce-dda5-4201-96e7-2804674a8ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:20.297813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05bcce-dda5-4201-96e7-2804674a8ed5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:20.304335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-df05bcce-dda5-4201-96e7-2804674a8ed5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:20.309294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:20.310313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:20.311217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 266/524] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:20.311772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05bcce-dda5-4201-96e7-2804674a8ed5 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:20.336593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaab0f7-087d-40c4-9a0b-9d2bca257d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:20.337268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaab0f7-087d-40c4-9a0b-9d2bca257d95 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:20.337552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaab0f7-087d-40c4-9a0b-9d2bca257d95 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:20.337829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaab0f7-087d-40c4-9a0b-9d2bca257d95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:20.345086 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaab0f7-087d-40c4-9a0b-9d2bca257d95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:20.346545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaab0f7-087d-40c4-9a0b-9d2bca257d95 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-ecaab0f7-087d-40c4-9a0b-9d2bca257d95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:20.359991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:20.361121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:20.367019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:20.367019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:20.367019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:20.367019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:20.367019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:20.367019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:20.367019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:20.367019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:20.367019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 259/525] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:20.367019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaab0f7-087d-40c4-9a0b-9d2bca257d95 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:20.428599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05bcce-dda5-4201-96e7-2804674a8ed5 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:20.466977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026357-4944-4cc5-80b3-d52f1e2e748d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:20.468121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026357-4944-4cc5-80b3-d52f1e2e748d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:20.468476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026357-4944-4cc5-80b3-d52f1e2e748d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:20.468852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026357-4944-4cc5-80b3-d52f1e2e748d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:20.473542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaab0f7-087d-40c4-9a0b-9d2bca257d95 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:20.476506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8026357-4944-4cc5-80b3-d52f1e2e748d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:20.478157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8026357-4944-4cc5-80b3-d52f1e2e748d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-c8026357-4944-4cc5-80b3-d52f1e2e748d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:20.490582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:20.491767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:20.492934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:20.492934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:20.492934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:20.492934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:20.492934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:20.492934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:20.492934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:20.492934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:20.498788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 267/526] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:20.498788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026357-4944-4cc5-80b3-d52f1e2e748d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:20.517399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9ab6b-0ba4-49bd-8f64-f0ddd74e5f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:20.518023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9ab6b-0ba4-49bd-8f64-f0ddd74e5f7d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:20.518300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9ab6b-0ba4-49bd-8f64-f0ddd74e5f7d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:20.518570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9ab6b-0ba4-49bd-8f64-f0ddd74e5f7d None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:20.519105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:20.519216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9ab6b-0ba4-49bd-8f64-f0ddd74e5f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:20.519508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9ab6b-0ba4-49bd-8f64-f0ddd74e5f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:20.520886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9ab6b-0ba4-49bd-8f64-f0ddd74e5f7d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:20.533426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 260/527] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:20.536294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9ab6b-0ba4-49bd-8f64-f0ddd74e5f7d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:20.562488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:20.628065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026357-4944-4cc5-80b3-d52f1e2e748d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:20.640171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9ab6b-0ba4-49bd-8f64-f0ddd74e5f7d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:20.669387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f5af1-15fc-4599-8a7f-b9aaa1fd7c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:20.670275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f5af1-15fc-4599-8a7f-b9aaa1fd7c63 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:20.670710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f5af1-15fc-4599-8a7f-b9aaa1fd7c63 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:20.670988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f5af1-15fc-4599-8a7f-b9aaa1fd7c63 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:20.671875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f5af1-15fc-4599-8a7f-b9aaa1fd7c63 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:20.672185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f5af1-15fc-4599-8a7f-b9aaa1fd7c63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:20.673756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f5af1-15fc-4599-8a7f-b9aaa1fd7c63 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:20.675050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ab285-3bc8-4490-9342-0a5b079231b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:20.675716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ab285-3bc8-4490-9342-0a5b079231b0 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:20.676058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ab285-3bc8-4490-9342-0a5b079231b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:20.676396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ab285-3bc8-4490-9342-0a5b079231b0 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:20.677185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ab285-3bc8-4490-9342-0a5b079231b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:20.677554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ab285-3bc8-4490-9342-0a5b079231b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:20.679713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ab285-3bc8-4490-9342-0a5b079231b0 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:20.697941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 268/528] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 8 18:33:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:20.697941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f5af1-15fc-4599-8a7f-b9aaa1fd7c63 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:20.698235 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 261/529] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:20.698976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ab285-3bc8-4490-9342-0a5b079231b0 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:20.785093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:20.793446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:20.808318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ab285-3bc8-4490-9342-0a5b079231b0 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:20.817631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f5af1-15fc-4599-8a7f-b9aaa1fd7c63 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:20.850189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495634f-887c-4e1f-b9eb-54054fb89370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:20.851392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495634f-887c-4e1f-b9eb-54054fb89370 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:20.851680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495634f-887c-4e1f-b9eb-54054fb89370 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:20.851952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495634f-887c-4e1f-b9eb-54054fb89370 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:20.852595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca27e5-ebe3-464f-8121-6a2c1530e739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:20.853294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca27e5-ebe3-464f-8121-6a2c1530e739 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:20.853637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca27e5-ebe3-464f-8121-6a2c1530e739 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:20.853975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca27e5-ebe3-464f-8121-6a2c1530e739 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:20.854739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca27e5-ebe3-464f-8121-6a2c1530e739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:20.855112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca27e5-ebe3-464f-8121-6a2c1530e739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:20.856727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca27e5-ebe3-464f-8121-6a2c1530e739 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:20.862303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495634f-887c-4e1f-b9eb-54054fb89370 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:20.863764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495634f-887c-4e1f-b9eb-54054fb89370 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-d495634f-887c-4e1f-b9eb-54054fb89370 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:20.875299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:20.876338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:20.882486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:20.882486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:20.882486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:20.882486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:20.882486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:20.882486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:20.882486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:20.882486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:20.882486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 262/530] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:20.882486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495634f-887c-4e1f-b9eb-54054fb89370 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:20.892413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 269/531] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:20.892413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca27e5-ebe3-464f-8121-6a2c1530e739 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:20.997452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca27e5-ebe3-464f-8121-6a2c1530e739 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:21.029964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495634f-887c-4e1f-b9eb-54054fb89370 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:21.032303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a987311-81d1-45f5-b116-25fc234672b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:21.032931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a987311-81d1-45f5-b116-25fc234672b7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:21.034137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a987311-81d1-45f5-b116-25fc234672b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:21.034423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a987311-81d1-45f5-b116-25fc234672b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:21.035160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a987311-81d1-45f5-b116-25fc234672b7 None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=1b29c5745a8e42ff8232e5ca22b4a4ad) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:21.035475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a987311-81d1-45f5-b116-25fc234672b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:21.036908 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a987311-81d1-45f5-b116-25fc234672b7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:21.039445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:21.045487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 270/532] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:20 2022] GET /identity/v3/role_assignments?user.id=1b29c5745a8e42ff8232e5ca22b4a4ad&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:21.058146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a987311-81d1-45f5-b116-25fc234672b7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:21.081629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798cc4e-57e2-4a7a-9ef7-3b441a447d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:21.082433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798cc4e-57e2-4a7a-9ef7-3b441a447d3f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:21.082819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798cc4e-57e2-4a7a-9ef7-3b441a447d3f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:21.083855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798cc4e-57e2-4a7a-9ef7-3b441a447d3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:21.109446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:21.177580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a987311-81d1-45f5-b116-25fc234672b7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:21.183142 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798cc4e-57e2-4a7a-9ef7-3b441a447d3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:21.184794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798cc4e-57e2-4a7a-9ef7-3b441a447d3f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:21.220170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148c614-2bb5-4f9a-85cb-9b0133af6e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:21.220795 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148c614-2bb5-4f9a-85cb-9b0133af6e15 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:21.221101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148c614-2bb5-4f9a-85cb-9b0133af6e15 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:21.221397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148c614-2bb5-4f9a-85cb-9b0133af6e15 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:21.222094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148c614-2bb5-4f9a-85cb-9b0133af6e15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:21.222408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148c614-2bb5-4f9a-85cb-9b0133af6e15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:21.223866 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148c614-2bb5-4f9a-85cb-9b0133af6e15 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:21.240811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 271/533] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:21.246241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148c614-2bb5-4f9a-85cb-9b0133af6e15 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:21.317884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 263/534] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:21.319070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798cc4e-57e2-4a7a-9ef7-3b441a447d3f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:21.320132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:21.357691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148c614-2bb5-4f9a-85cb-9b0133af6e15 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:21.373191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:21.402376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ba5b4-397e-4c8a-a65b-8b16e042b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:21.402874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ba5b4-397e-4c8a-a65b-8b16e042b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:21.403159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ba5b4-397e-4c8a-a65b-8b16e042b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:21.403433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ba5b4-397e-4c8a-a65b-8b16e042b5b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:21.404163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ba5b4-397e-4c8a-a65b-8b16e042b5b9 None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=605b5d4c0fc9409a94dad22c77c67332) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:21.404505 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ba5b4-397e-4c8a-a65b-8b16e042b5b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:21.406704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ba5b4-397e-4c8a-a65b-8b16e042b5b9 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:21.417321 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 272/535] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:21 2022] GET /identity/v3/role_assignments?user.id=605b5d4c0fc9409a94dad22c77c67332&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:21.418295 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ba5b4-397e-4c8a-a65b-8b16e042b5b9 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:21.443824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798cc4e-57e2-4a7a-9ef7-3b441a447d3f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:21.491040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13699fd5-758f-495d-a01c-30fe9330d779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:21.492719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13699fd5-758f-495d-a01c-30fe9330d779 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:21.493178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13699fd5-758f-495d-a01c-30fe9330d779 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:21.493731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13699fd5-758f-495d-a01c-30fe9330d779 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:21.494743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13699fd5-758f-495d-a01c-30fe9330d779 None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=c8d10585480f41608bc6935e282f7e27) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:21.495402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13699fd5-758f-495d-a01c-30fe9330d779 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:21.497019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13699fd5-758f-495d-a01c-30fe9330d779 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:21.504726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 264/536] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:21 2022] GET /identity/v3/role_assignments?user.id=c8d10585480f41608bc6935e282f7e27&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:21.595277 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ba5b4-397e-4c8a-a65b-8b16e042b5b9 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:21.624112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:21.638658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95306f9-cc1d-4589-b3c3-cb5989584377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:21.639523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95306f9-cc1d-4589-b3c3-cb5989584377 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:21.639952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95306f9-cc1d-4589-b3c3-cb5989584377 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:21.640330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95306f9-cc1d-4589-b3c3-cb5989584377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:21.654257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95306f9-cc1d-4589-b3c3-cb5989584377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:21.655982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95306f9-cc1d-4589-b3c3-cb5989584377 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-f95306f9-cc1d-4589-b3c3-cb5989584377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:21.672923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:21.673916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:21.674768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:21.674768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:21.674768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:21.674768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 273/537] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:21.680308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13699fd5-758f-495d-a01c-30fe9330d779 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:21.802431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13699fd5-758f-495d-a01c-30fe9330d779 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:21.814362 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:21.845683 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a96ee-79c0-4b8b-806b-a709f30e9f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:21.846658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a96ee-79c0-4b8b-806b-a709f30e9f7f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:21.847137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a96ee-79c0-4b8b-806b-a709f30e9f7f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:21.847603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a96ee-79c0-4b8b-806b-a709f30e9f7f None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:21.848471 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a96ee-79c0-4b8b-806b-a709f30e9f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:21.848984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a96ee-79c0-4b8b-806b-a709f30e9f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:21.850610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a96ee-79c0-4b8b-806b-a709f30e9f7f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:21.865320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 265/538] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:21.871077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95306f9-cc1d-4589-b3c3-cb5989584377 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:22.025438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95306f9-cc1d-4589-b3c3-cb5989584377 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:22.084902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b3106-854d-499a-a36c-b307ef4d0d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:22.085810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b3106-854d-499a-a36c-b307ef4d0d41 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:22.086100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b3106-854d-499a-a36c-b307ef4d0d41 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:22.086374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b3106-854d-499a-a36c-b307ef4d0d41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:22.100166 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b3106-854d-499a-a36c-b307ef4d0d41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:22.101707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b3106-854d-499a-a36c-b307ef4d0d41 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-e78b3106-854d-499a-a36c-b307ef4d0d41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:22.116175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:22.117974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:22.118823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:22.118823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:22.118823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:22.118823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:22.118823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:22.118823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:22.118823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:22.118823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:22.122888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 274/539] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:22.125752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a96ee-79c0-4b8b-806b-a709f30e9f7f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:22.290385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a96ee-79c0-4b8b-806b-a709f30e9f7f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:22.341390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02bbc1-04ed-4026-a3f8-ee1848992c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:22.342389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02bbc1-04ed-4026-a3f8-ee1848992c23 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:22.343122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02bbc1-04ed-4026-a3f8-ee1848992c23 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:22.343593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02bbc1-04ed-4026-a3f8-ee1848992c23 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:22.344511 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02bbc1-04ed-4026-a3f8-ee1848992c23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:22.345028 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02bbc1-04ed-4026-a3f8-ee1848992c23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:22.346764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02bbc1-04ed-4026-a3f8-ee1848992c23 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:22.355947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b43b8-6b8e-4e88-8f4b-523ebe3b13af None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:22.356415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b43b8-6b8e-4e88-8f4b-523ebe3b13af None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:22.356415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b43b8-6b8e-4e88-8f4b-523ebe3b13af None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:22.358084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 275/540] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:33:22.363014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e3279-6e27-4642-88ce-dbd0a159c0e9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:22.363308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e3279-6e27-4642-88ce-dbd0a159c0e9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:22.365922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e3279-6e27-4642-88ce-dbd0a159c0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:22.370598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 266/541] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:22.376567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02bbc1-04ed-4026-a3f8-ee1848992c23 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:22.394047 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-0c8e3279-6e27-4642-88ce-dbd0a159c0e9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:22.395260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0c8e3279-6e27-4642-88ce-dbd0a159c0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:22.424296 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:22.455546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:22.532639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 276/542] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:33:22.542692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02bbc1-04ed-4026-a3f8-ee1848992c23 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:22.586482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7c98c-b67c-482f-811e-606bcd927216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:22.587514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7c98c-b67c-482f-811e-606bcd927216 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:22.588008 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7c98c-b67c-482f-811e-606bcd927216 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:22.588511 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7c98c-b67c-482f-811e-606bcd927216 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:22.589462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7c98c-b67c-482f-811e-606bcd927216 None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=bcf30ef695154d3a97d7e589aca21233) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:22.589986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7c98c-b67c-482f-811e-606bcd927216 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:22.591641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7c98c-b67c-482f-811e-606bcd927216 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:22.598237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 267/543] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:22 2022] GET /identity/v3/role_assignments?user.id=bcf30ef695154d3a97d7e589aca21233&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:22.850157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e3279-6e27-4642-88ce-dbd0a159c0e9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:23.008786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e3279-6e27-4642-88ce-dbd0a159c0e9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:23.057783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b9537-1824-4059-824b-b671e379020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:23.058450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b9537-1824-4059-824b-b671e379020c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:23.058734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b9537-1824-4059-824b-b671e379020c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:23.059013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b9537-1824-4059-824b-b671e379020c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:23.073964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b9537-1824-4059-824b-b671e379020c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-383b9537-1824-4059-824b-b671e379020c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:23.078876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:23.079752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:23.080555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 277/544] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:23.087185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7c98c-b67c-482f-811e-606bcd927216 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:23.242936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:23.255199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7c98c-b67c-482f-811e-606bcd927216 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:23.306894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9207d-c4db-4d3b-95e8-c69c9b0b625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:23.307512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9207d-c4db-4d3b-95e8-c69c9b0b625e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:23.307766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9207d-c4db-4d3b-95e8-c69c9b0b625e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:23.308037 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9207d-c4db-4d3b-95e8-c69c9b0b625e None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:23.308699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9207d-c4db-4d3b-95e8-c69c9b0b625e None admin] RBAC: Adding query filter params (name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:23.309004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9207d-c4db-4d3b-95e8-c69c9b0b625e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:23.310392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9207d-c4db-4d3b-95e8-c69c9b0b625e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:23.325286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 268/545] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:23.330747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b9537-1824-4059-824b-b671e379020c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:23.380577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:23.494899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b9537-1824-4059-824b-b671e379020c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:23.547685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82bd3f-e75b-4943-bf5b-24bece310f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:23.548329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82bd3f-e75b-4943-bf5b-24bece310f66 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:23.548614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82bd3f-e75b-4943-bf5b-24bece310f66 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:23.548900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82bd3f-e75b-4943-bf5b-24bece310f66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:23.562399 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82bd3f-e75b-4943-bf5b-24bece310f66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:23.563855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82bd3f-e75b-4943-bf5b-24bece310f66 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-6f82bd3f-e75b-4943-bf5b-24bece310f66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:23.585153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:23.586281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:23.587123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:23.587123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:23.587123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:23.587123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 278/546] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:23.595412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9207d-c4db-4d3b-95e8-c69c9b0b625e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:23.711879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9207d-c4db-4d3b-95e8-c69c9b0b625e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:23.748294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455e472-e21f-4972-a32f-2b8e8560b881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:23.748832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455e472-e21f-4972-a32f-2b8e8560b881 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:23.749143 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455e472-e21f-4972-a32f-2b8e8560b881 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:23.749422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455e472-e21f-4972-a32f-2b8e8560b881 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:23.750127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455e472-e21f-4972-a32f-2b8e8560b881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:23.750436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455e472-e21f-4972-a32f-2b8e8560b881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:23.751816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455e472-e21f-4972-a32f-2b8e8560b881 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:23.764980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 269/547] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:23.774782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82bd3f-e75b-4943-bf5b-24bece310f66 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:23.800194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cae03-3c35-4922-82c9-436f69f714ff None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:23.800513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cae03-3c35-4922-82c9-436f69f714ff None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:23.800760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cae03-3c35-4922-82c9-436f69f714ff None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:23.802370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 270/548] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:23.808429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1eed4-9b10-473c-a6ac-fc5d195ccb4a None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:23.808741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1eed4-9b10-473c-a6ac-fc5d195ccb4a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:23.808983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1eed4-9b10-473c-a6ac-fc5d195ccb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:23.828219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-18f1eed4-9b10-473c-a6ac-fc5d195ccb4a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:23.829461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-18f1eed4-9b10-473c-a6ac-fc5d195ccb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:23.842089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:23.904092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 271/549] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:33:23.975103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:24.011271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82bd3f-e75b-4943-bf5b-24bece310f66 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:24.055553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905121ac-d9d3-4e9c-b093-e96fc571095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:24.056207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905121ac-d9d3-4e9c-b093-e96fc571095c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:24.056492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905121ac-d9d3-4e9c-b093-e96fc571095c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:24.056774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905121ac-d9d3-4e9c-b093-e96fc571095c None admin] PATH_INFO: `/v3/users/swift` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:24.068365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905121ac-d9d3-4e9c-b093-e96fc571095c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:24.069964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905121ac-d9d3-4e9c-b093-e96fc571095c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-905121ac-d9d3-4e9c-b093-e96fc571095c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:24.084665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:24.085734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:24.086641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:24.086641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:24.086641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:24.086641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:24.086641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:24.086641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:24.086641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:24.086641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:24.086641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:24.086641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:24.089681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 279/550] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:24.092304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1eed4-9b10-473c-a6ac-fc5d195ccb4a None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:24.247269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1eed4-9b10-473c-a6ac-fc5d195ccb4a None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:24.271611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905121ac-d9d3-4e9c-b093-e96fc571095c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:24.285915 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adb0e2-5c04-4cb5-bfdb-12eee0ba7e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:24.286593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adb0e2-5c04-4cb5-bfdb-12eee0ba7e91 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:24.286885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adb0e2-5c04-4cb5-bfdb-12eee0ba7e91 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:24.287228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adb0e2-5c04-4cb5-bfdb-12eee0ba7e91 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:24.287991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adb0e2-5c04-4cb5-bfdb-12eee0ba7e91 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:24.288849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adb0e2-5c04-4cb5-bfdb-12eee0ba7e91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:24.290872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adb0e2-5c04-4cb5-bfdb-12eee0ba7e91 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:24.312716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 272/551] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 18:33:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:24.322546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adb0e2-5c04-4cb5-bfdb-12eee0ba7e91 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:24.342612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:24.433161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adb0e2-5c04-4cb5-bfdb-12eee0ba7e91 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:24.456097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905121ac-d9d3-4e9c-b093-e96fc571095c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:24.468855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b3619-0792-459e-8bae-01172f38111d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:24.469550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3619-0792-459e-8bae-01172f38111d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:24.469832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3619-0792-459e-8bae-01172f38111d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:24.470105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3619-0792-459e-8bae-01172f38111d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:24.493900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7baa41-7f37-44f9-9a4b-b1794b4e1bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:24.494534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7baa41-7f37-44f9-9a4b-b1794b4e1bda None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:24.495078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7baa41-7f37-44f9-9a4b-b1794b4e1bda None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:24.495361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7baa41-7f37-44f9-9a4b-b1794b4e1bda None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:24.499056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:24.509467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7baa41-7f37-44f9-9a4b-b1794b4e1bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-6f7baa41-7f37-44f9-9a4b-b1794b4e1bda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:24.515061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:24.515955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:24.516852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:24.517539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 280/552] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:24.518583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1b2f1-447f-4169-a8c0-a4bc2d55a49d None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:24.518864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1b2f1-447f-4169-a8c0-a4bc2d55a49d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:24.519157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1b2f1-447f-4169-a8c0-a4bc2d55a49d None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:24.525998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 281/553] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:24.525998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c412f-fc91-41aa-840d-baf8176aebf5 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:24.525998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c412f-fc91-41aa-840d-baf8176aebf5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:24.525998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c412f-fc91-41aa-840d-baf8176aebf5 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:24.527792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 282/554] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:24.528643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c412f-fc91-41aa-840d-baf8176aebf5 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:24.552663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b3619-0792-459e-8bae-01172f38111d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:24.555121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b3619-0792-459e-8bae-01172f38111d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:24.558413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:24.641417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c412f-fc91-41aa-840d-baf8176aebf5 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:24.673938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 273/555] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 18:33:24.673938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901ce65-ad9a-4e13-b74a-480a01631747 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:24.673938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901ce65-ad9a-4e13-b74a-480a01631747 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:24.673938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901ce65-ad9a-4e13-b74a-480a01631747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:24.681030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edfa0b-d122-4fa4-8971-8d5b0301a662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:24.681765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edfa0b-d122-4fa4-8971-8d5b0301a662 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:24.682049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edfa0b-d122-4fa4-8971-8d5b0301a662 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:24.682325 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edfa0b-d122-4fa4-8971-8d5b0301a662 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:24.683011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edfa0b-d122-4fa4-8971-8d5b0301a662 None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:24.683320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edfa0b-d122-4fa4-8971-8d5b0301a662 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:24.684694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edfa0b-d122-4fa4-8971-8d5b0301a662 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:24.690723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-3901ce65-ad9a-4e13-b74a-480a01631747 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:24.691944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3901ce65-ad9a-4e13-b74a-480a01631747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:24.697565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 283/556] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:24.697565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c974369-ca21-457e-8181-565d139b99cf None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:24.697565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c974369-ca21-457e-8181-565d139b99cf None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:24.697565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c974369-ca21-457e-8181-565d139b99cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:24.713126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8c974369-ca21-457e-8181-565d139b99cf None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:24.714479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8c974369-ca21-457e-8181-565d139b99cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:24.772434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:24.777447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 274/557] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:24.778380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901ce65-ad9a-4e13-b74a-480a01631747 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:24.797308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 284/558] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:24.799245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c974369-ca21-457e-8181-565d139b99cf None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:24.838762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:24.949771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901ce65-ad9a-4e13-b74a-480a01631747 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.002269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69745080-a715-447b-ad47-185c14748f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.003205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745080-a715-447b-ad47-185c14748f73 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.003668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745080-a715-447b-ad47-185c14748f73 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.004142 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745080-a715-447b-ad47-185c14748f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.015770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69745080-a715-447b-ad47-185c14748f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:25.017484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69745080-a715-447b-ad47-185c14748f73 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-69745080-a715-447b-ad47-185c14748f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:25.033774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:25.034703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:25.035589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:25.035589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:25.035589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:25.035766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c974369-ca21-457e-8181-565d139b99cf None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.037524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 275/559] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:25.044857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69745080-a715-447b-ad47-185c14748f73 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.087211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a40d8-56e1-463b-b98a-046ecd537cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.087921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a40d8-56e1-463b-b98a-046ecd537cd4 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.088343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a40d8-56e1-463b-b98a-046ecd537cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.088837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a40d8-56e1-463b-b98a-046ecd537cd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.099894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a40d8-56e1-463b-b98a-046ecd537cd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:25.101694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a40d8-56e1-463b-b98a-046ecd537cd4 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-a99a40d8-56e1-463b-b98a-046ecd537cd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:25.116967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:25.118240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:25.119215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:25.119215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:25.119215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:25.119215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 285/560] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:25.119215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a40d8-56e1-463b-b98a-046ecd537cd4 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.151807 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:25.160497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:25.167046 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69745080-a715-447b-ad47-185c14748f73 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.201033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fabbe0-ff55-4bda-ae69-8d1cfeb1650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.201678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fabbe0-ff55-4bda-ae69-8d1cfeb1650f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.201959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fabbe0-ff55-4bda-ae69-8d1cfeb1650f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.202235 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fabbe0-ff55-4bda-ae69-8d1cfeb1650f None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.202895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fabbe0-ff55-4bda-ae69-8d1cfeb1650f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:25.203189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fabbe0-ff55-4bda-ae69-8d1cfeb1650f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:25.204576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fabbe0-ff55-4bda-ae69-8d1cfeb1650f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:25.216928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 276/561] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:25.218418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698731f-d0c2-480e-8f50-4c0112e08893 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.218699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698731f-d0c2-480e-8f50-4c0112e08893 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.218971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698731f-d0c2-480e-8f50-4c0112e08893 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.221097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 277/562] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:25.221222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698731f-d0c2-480e-8f50-4c0112e08893 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.223869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a40d8-56e1-463b-b98a-046ecd537cd4 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.258550 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbed33-0e3c-4c53-a2a3-28dd500a9ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.259177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbed33-0e3c-4c53-a2a3-28dd500a9ee0 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.259467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbed33-0e3c-4c53-a2a3-28dd500a9ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.259745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbed33-0e3c-4c53-a2a3-28dd500a9ee0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.270063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbed33-0e3c-4c53-a2a3-28dd500a9ee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-6cbbed33-0e3c-4c53-a2a3-28dd500a9ee0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:25.276173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:25.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:25.278007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 286/563] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:25.278699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbed33-0e3c-4c53-a2a3-28dd500a9ee0 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.344820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698731f-d0c2-480e-8f50-4c0112e08893 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.375050 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:25.378134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbed33-0e3c-4c53-a2a3-28dd500a9ee0 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.379962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7fcba-81c7-4861-9f60-7f7b47d00709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.380636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7fcba-81c7-4861-9f60-7f7b47d00709 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.380918 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7fcba-81c7-4861-9f60-7f7b47d00709 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.381219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7fcba-81c7-4861-9f60-7f7b47d00709 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.385902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7fcba-81c7-4861-9f60-7f7b47d00709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:25.385902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7fcba-81c7-4861-9f60-7f7b47d00709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:25.385902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7fcba-81c7-4861-9f60-7f7b47d00709 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:25.395718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 278/564] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:25.396508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7fcba-81c7-4861-9f60-7f7b47d00709 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.405139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:25.414206 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838682d-79c8-49b1-8abd-c673f7533513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.414955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838682d-79c8-49b1-8abd-c673f7533513 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.415232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838682d-79c8-49b1-8abd-c673f7533513 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.415507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838682d-79c8-49b1-8abd-c673f7533513 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.427483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838682d-79c8-49b1-8abd-c673f7533513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-6838682d-79c8-49b1-8abd-c673f7533513 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:25.433093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:25.434214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:25.435070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 287/565] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:25.435766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2926d04-da71-4982-bb35-6a7a2119fa25 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.436403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2926d04-da71-4982-bb35-6a7a2119fa25 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.436678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2926d04-da71-4982-bb35-6a7a2119fa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.454051 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d2926d04-da71-4982-bb35-6a7a2119fa25 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:25.455278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d2926d04-da71-4982-bb35-6a7a2119fa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:25.500771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7fcba-81c7-4861-9f60-7f7b47d00709 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.522758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 288/566] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:25.523552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2926d04-da71-4982-bb35-6a7a2119fa25 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.555643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2626-c0ac-4627-b105-008dc6e77196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.556634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2626-c0ac-4627-b105-008dc6e77196 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.556926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2626-c0ac-4627-b105-008dc6e77196 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.557233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2626-c0ac-4627-b105-008dc6e77196 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.566120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af2626-c0ac-4627-b105-008dc6e77196 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:25.568239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af2626-c0ac-4627-b105-008dc6e77196 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-b2af2626-c0ac-4627-b105-008dc6e77196 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:25.583638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:25.584556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:25.591195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:25.591195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:25.591195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:25.591195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:25.591195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:25.591195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:25.591195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:25.591195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:25.591195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 279/567] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:25.591195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2626-c0ac-4627-b105-008dc6e77196 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.630383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:25.642157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2926d04-da71-4982-bb35-6a7a2119fa25 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.666969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:25.687932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cacbb-ff30-4d14-a1a6-0cfedeb9de41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.688561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cacbb-ff30-4d14-a1a6-0cfedeb9de41 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.688821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cacbb-ff30-4d14-a1a6-0cfedeb9de41 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.689130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cacbb-ff30-4d14-a1a6-0cfedeb9de41 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.689798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cacbb-ff30-4d14-a1a6-0cfedeb9de41 None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:25.690101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cacbb-ff30-4d14-a1a6-0cfedeb9de41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:25.691447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cacbb-ff30-4d14-a1a6-0cfedeb9de41 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:25.704173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2626-c0ac-4627-b105-008dc6e77196 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.704811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 289/568] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:25.705704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cacbb-ff30-4d14-a1a6-0cfedeb9de41 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.742252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06190ed-e30f-4a8d-8eda-de1dec4acfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.742878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06190ed-e30f-4a8d-8eda-de1dec4acfe6 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.743159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06190ed-e30f-4a8d-8eda-de1dec4acfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.743426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06190ed-e30f-4a8d-8eda-de1dec4acfe6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.751077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06190ed-e30f-4a8d-8eda-de1dec4acfe6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:25.752986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06190ed-e30f-4a8d-8eda-de1dec4acfe6 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-a06190ed-e30f-4a8d-8eda-de1dec4acfe6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:25.765447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:25.766433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 280/569] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:25.767378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06190ed-e30f-4a8d-8eda-de1dec4acfe6 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.816464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cacbb-ff30-4d14-a1a6-0cfedeb9de41 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.851863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba73da-0544-4672-af49-126ff0c9982b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.852474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba73da-0544-4672-af49-126ff0c9982b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.852757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba73da-0544-4672-af49-126ff0c9982b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.853032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba73da-0544-4672-af49-126ff0c9982b None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.853719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba73da-0544-4672-af49-126ff0c9982b None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:25.854023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba73da-0544-4672-af49-126ff0c9982b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:25.855380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba73da-0544-4672-af49-126ff0c9982b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:25.863746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 290/570] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:25.864487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba73da-0544-4672-af49-126ff0c9982b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.873483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06190ed-e30f-4a8d-8eda-de1dec4acfe6 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:25.880087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:25.904171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:25.912690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89329ca9-6dd2-4374-bc85-f51ce1d137c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:25.913320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89329ca9-6dd2-4374-bc85-f51ce1d137c4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:25.913580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89329ca9-6dd2-4374-bc85-f51ce1d137c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:25.913860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89329ca9-6dd2-4374-bc85-f51ce1d137c4 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:25.914542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89329ca9-6dd2-4374-bc85-f51ce1d137c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:25.915155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89329ca9-6dd2-4374-bc85-f51ce1d137c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:25.916289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89329ca9-6dd2-4374-bc85-f51ce1d137c4 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:25.927878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 281/571] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:25.929608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89329ca9-6dd2-4374-bc85-f51ce1d137c4 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:25.976538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba73da-0544-4672-af49-126ff0c9982b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.010324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915ffae-10c2-49d2-a99e-225aebb2e7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.010944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915ffae-10c2-49d2-a99e-225aebb2e7d3 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.011218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915ffae-10c2-49d2-a99e-225aebb2e7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.011489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915ffae-10c2-49d2-a99e-225aebb2e7d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.021418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915ffae-10c2-49d2-a99e-225aebb2e7d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.023376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915ffae-10c2-49d2-a99e-225aebb2e7d3 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.033108 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89329ca9-6dd2-4374-bc85-f51ce1d137c4 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-6915ffae-10c2-49d2-a99e-225aebb2e7d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:26.038105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:26.039622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:26.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:26.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:26.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:26.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 291/572] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:26.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915ffae-10c2-49d2-a99e-225aebb2e7d3 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.070687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f82d4-0d50-4b77-81fa-44edbc542917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.071293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f82d4-0d50-4b77-81fa-44edbc542917 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.071573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f82d4-0d50-4b77-81fa-44edbc542917 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.071846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f82d4-0d50-4b77-81fa-44edbc542917 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.072569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f82d4-0d50-4b77-81fa-44edbc542917 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:26.072878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f82d4-0d50-4b77-81fa-44edbc542917 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.074645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f82d4-0d50-4b77-81fa-44edbc542917 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.092778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 282/573] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 18:33:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:26.093887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f82d4-0d50-4b77-81fa-44edbc542917 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.104457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:26.139647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:26.140071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915ffae-10c2-49d2-a99e-225aebb2e7d3 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.174595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d1050-1c0c-4694-9107-29614b7c7f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.175499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d1050-1c0c-4694-9107-29614b7c7f41 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.175788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d1050-1c0c-4694-9107-29614b7c7f41 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.176062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d1050-1c0c-4694-9107-29614b7c7f41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.186894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d1050-1c0c-4694-9107-29614b7c7f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-ff1d1050-1c0c-4694-9107-29614b7c7f41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:26.192048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:26.193028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 292/574] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:26.193928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d1050-1c0c-4694-9107-29614b7c7f41 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.195496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f82d4-0d50-4b77-81fa-44edbc542917 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.230598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccd05a-9fb9-48de-bc68-3ad0770781d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.231183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccd05a-9fb9-48de-bc68-3ad0770781d7 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.231470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccd05a-9fb9-48de-bc68-3ad0770781d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.231739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccd05a-9fb9-48de-bc68-3ad0770781d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.244239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccd05a-9fb9-48de-bc68-3ad0770781d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.245737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccd05a-9fb9-48de-bc68-3ad0770781d7 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-2bccd05a-9fb9-48de-bc68-3ad0770781d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:26.263050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:26.264047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:26.264926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:26.264926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:26.264926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:26.264926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 283/575] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:26.264926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccd05a-9fb9-48de-bc68-3ad0770781d7 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.313845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d1050-1c0c-4694-9107-29614b7c7f41 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.339014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:26.347797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538e18a-4128-494b-b27a-bd48d7b8219d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.348404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538e18a-4128-494b-b27a-bd48d7b8219d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.348685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538e18a-4128-494b-b27a-bd48d7b8219d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.348959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538e18a-4128-494b-b27a-bd48d7b8219d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.349713 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538e18a-4128-494b-b27a-bd48d7b8219d None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=88a9b2cee21f4831b221d113ddbbd29d) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:26.350042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538e18a-4128-494b-b27a-bd48d7b8219d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.352142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538e18a-4128-494b-b27a-bd48d7b8219d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.357800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 293/576] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/role_assignments?user.id=88a9b2cee21f4831b221d113ddbbd29d&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:26.358626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538e18a-4128-494b-b27a-bd48d7b8219d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.372556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:26.380230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccd05a-9fb9-48de-bc68-3ad0770781d7 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.423781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87efb8-452f-4568-8064-ae6d37a1b072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.424310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87efb8-452f-4568-8064-ae6d37a1b072 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.424589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87efb8-452f-4568-8064-ae6d37a1b072 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.424861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87efb8-452f-4568-8064-ae6d37a1b072 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.425567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87efb8-452f-4568-8064-ae6d37a1b072 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:26.425870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87efb8-452f-4568-8064-ae6d37a1b072 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.427273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87efb8-452f-4568-8064-ae6d37a1b072 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.444906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 284/577] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:26.444906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87efb8-452f-4568-8064-ae6d37a1b072 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.487562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538e18a-4128-494b-b27a-bd48d7b8219d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.527197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994688e4-5c48-490a-b429-21ed3de10b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.527877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994688e4-5c48-490a-b429-21ed3de10b83 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.528099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994688e4-5c48-490a-b429-21ed3de10b83 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.528375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994688e4-5c48-490a-b429-21ed3de10b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.541661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994688e4-5c48-490a-b429-21ed3de10b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.543101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994688e4-5c48-490a-b429-21ed3de10b83 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.558996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87efb8-452f-4568-8064-ae6d37a1b072 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-994688e4-5c48-490a-b429-21ed3de10b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:26.562185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:26.563126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:26.564025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:26.564025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:26.564025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:26.568354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 294/578] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:26.569306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994688e4-5c48-490a-b429-21ed3de10b83 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.607514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ddcbd-afae-473d-ac38-bf8dc69ef625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.608153 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ddcbd-afae-473d-ac38-bf8dc69ef625 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.608435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ddcbd-afae-473d-ac38-bf8dc69ef625 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.608710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ddcbd-afae-473d-ac38-bf8dc69ef625 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.609430 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ddcbd-afae-473d-ac38-bf8dc69ef625 None admin] RBAC: Adding query filter params (name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:26.609739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ddcbd-afae-473d-ac38-bf8dc69ef625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.611141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ddcbd-afae-473d-ac38-bf8dc69ef625 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.624060 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 285/579] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:26.626162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ddcbd-afae-473d-ac38-bf8dc69ef625 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.648416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:26.656956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:26.721698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994688e4-5c48-490a-b429-21ed3de10b83 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.737067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ddcbd-afae-473d-ac38-bf8dc69ef625 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.757722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6b5f2-4daa-456e-a009-9338c9f5716d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.758337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6b5f2-4daa-456e-a009-9338c9f5716d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.758625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6b5f2-4daa-456e-a009-9338c9f5716d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.758904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6b5f2-4daa-456e-a009-9338c9f5716d None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.759544 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6b5f2-4daa-456e-a009-9338c9f5716d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:26.759851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6b5f2-4daa-456e-a009-9338c9f5716d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.761494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6b5f2-4daa-456e-a009-9338c9f5716d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.772152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09564d-3e74-4ee9-bf57-094ebc063a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.773341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09564d-3e74-4ee9-bf57-094ebc063a31 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.773624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09564d-3e74-4ee9-bf57-094ebc063a31 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.773890 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09564d-3e74-4ee9-bf57-094ebc063a31 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.774603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 295/580] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:26.775424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6b5f2-4daa-456e-a009-9338c9f5716d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.786299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09564d-3e74-4ee9-bf57-094ebc063a31 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.787800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09564d-3e74-4ee9-bf57-094ebc063a31 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-ff09564d-3e74-4ee9-bf57-094ebc063a31 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:26.801417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:26.802365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 286/581] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:26.803386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09564d-3e74-4ee9-bf57-094ebc063a31 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.893080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:26.896577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6b5f2-4daa-456e-a009-9338c9f5716d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.898884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:26.914411 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09564d-3e74-4ee9-bf57-094ebc063a31 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:26.943006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07caf75-d270-420d-b5a6-77080448f804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.943624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07caf75-d270-420d-b5a6-77080448f804 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.943908 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07caf75-d270-420d-b5a6-77080448f804 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.944182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07caf75-d270-420d-b5a6-77080448f804 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.945455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07caf75-d270-420d-b5a6-77080448f804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:26.945763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07caf75-d270-420d-b5a6-77080448f804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.947112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07caf75-d270-420d-b5a6-77080448f804 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.957598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd5861-548c-4795-9b23-a3beebe96743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:26.964270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd5861-548c-4795-9b23-a3beebe96743 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:26.964389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 296/582] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:26.964484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd5861-548c-4795-9b23-a3beebe96743 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:26.964484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd5861-548c-4795-9b23-a3beebe96743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:26.964637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07caf75-d270-420d-b5a6-77080448f804 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:26.975220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd5861-548c-4795-9b23-a3beebe96743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:26.976848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd5861-548c-4795-9b23-a3beebe96743 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-19dd5861-548c-4795-9b23-a3beebe96743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:26.991657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:26.992857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:26.993778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:26.993778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:26.993778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:26.993778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 287/583] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:26.993778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd5861-548c-4795-9b23-a3beebe96743 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.079909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07caf75-d270-420d-b5a6-77080448f804 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.103748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd5861-548c-4795-9b23-a3beebe96743 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.116449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2828e8-160a-44e9-84ae-e906f0e3ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:27.117207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2828e8-160a-44e9-84ae-e906f0e3ee89 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:27.117377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2828e8-160a-44e9-84ae-e906f0e3ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:27.117634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2828e8-160a-44e9-84ae-e906f0e3ee89 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:27.125349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2828e8-160a-44e9-84ae-e906f0e3ee89 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:27.126790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2828e8-160a-44e9-84ae-e906f0e3ee89 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:27.135776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-0f2828e8-160a-44e9-84ae-e906f0e3ee89 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:27.139649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:27.140551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 297/584] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:27.141547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2828e8-160a-44e9-84ae-e906f0e3ee89 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.147547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c405db-6d10-48cf-86b2-730122080379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:27.155504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c405db-6d10-48cf-86b2-730122080379 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:27.155504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c405db-6d10-48cf-86b2-730122080379 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:27.155504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c405db-6d10-48cf-86b2-730122080379 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:27.155504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c405db-6d10-48cf-86b2-730122080379 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:27.155504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c405db-6d10-48cf-86b2-730122080379 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:27.155504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c405db-6d10-48cf-86b2-730122080379 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:27.169061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:27.169975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 288/585] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 18:33:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:27.170747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c405db-6d10-48cf-86b2-730122080379 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.261480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2828e8-160a-44e9-84ae-e906f0e3ee89 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.287904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c405db-6d10-48cf-86b2-730122080379 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.296356 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6922b5-a6b4-4a8a-a313-81cde5c614ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:27.297096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6922b5-a6b4-4a8a-a313-81cde5c614ad None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:27.297367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6922b5-a6b4-4a8a-a313-81cde5c614ad None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:27.297638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6922b5-a6b4-4a8a-a313-81cde5c614ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:27.305298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6922b5-a6b4-4a8a-a313-81cde5c614ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:27.306739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6922b5-a6b4-4a8a-a313-81cde5c614ad None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-4f6922b5-a6b4-4a8a-a313-81cde5c614ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:27.319045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:27.320188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:27.321320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:27.321320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:27.321320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:27.321320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:27.321320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:27.321320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:27.321320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:27.321320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:27.324976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 298/586] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:27.325898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6922b5-a6b4-4a8a-a313-81cde5c614ad None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.327227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b91bf-6b5d-4b09-8b5e-3859452199f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:27.327904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b91bf-6b5d-4b09-8b5e-3859452199f1 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:27.328257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b91bf-6b5d-4b09-8b5e-3859452199f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:27.328602 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b91bf-6b5d-4b09-8b5e-3859452199f1 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:27.329347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b91bf-6b5d-4b09-8b5e-3859452199f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:27.329725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b91bf-6b5d-4b09-8b5e-3859452199f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:27.331193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b91bf-6b5d-4b09-8b5e-3859452199f1 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:27.344424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 289/587] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:27.345264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b91bf-6b5d-4b09-8b5e-3859452199f1 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.394277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:27.419716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:27.441980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6922b5-a6b4-4a8a-a313-81cde5c614ad None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.448450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b91bf-6b5d-4b09-8b5e-3859452199f1 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.481147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8ed40-2810-442c-b808-300b89f88ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:27.481784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8ed40-2810-442c-b808-300b89f88ecd None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:27.482068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8ed40-2810-442c-b808-300b89f88ecd None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:27.482342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8ed40-2810-442c-b808-300b89f88ecd None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:27.483062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8ed40-2810-442c-b808-300b89f88ecd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:27.483366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8ed40-2810-442c-b808-300b89f88ecd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:27.483978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7fbcc-4837-4ada-8866-6ec6085639a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:27.484600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7fbcc-4837-4ada-8866-6ec6085639a5 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:27.484774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8ed40-2810-442c-b808-300b89f88ecd None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:27.484869 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7fbcc-4837-4ada-8866-6ec6085639a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:27.485168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7fbcc-4837-4ada-8866-6ec6085639a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:27.496658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7fbcc-4837-4ada-8866-6ec6085639a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:27.498124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7fbcc-4837-4ada-8866-6ec6085639a5 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:27.508554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 299/588] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 18:33:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:27.509478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8ed40-2810-442c-b808-300b89f88ecd None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-59d7fbcc-4837-4ada-8866-6ec6085639a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:27.513269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:27.514406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:27.515285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:27.515285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:27.515285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:27.515285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 290/589] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:27.515521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7fbcc-4837-4ada-8866-6ec6085639a5 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.624845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8ed40-2810-442c-b808-300b89f88ecd None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.626078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:27.628757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7fbcc-4837-4ada-8866-6ec6085639a5 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.664070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6b815-cd16-4718-9c10-5a65f6d0c73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:27.664840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6b815-cd16-4718-9c10-5a65f6d0c73d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:27.665014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de66d33-a433-43a1-9ad6-cf05b40e520d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:27.665334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6b815-cd16-4718-9c10-5a65f6d0c73d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:27.665665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de66d33-a433-43a1-9ad6-cf05b40e520d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:27.665759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6b815-cd16-4718-9c10-5a65f6d0c73d None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:27.665940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de66d33-a433-43a1-9ad6-cf05b40e520d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:27.666207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de66d33-a433-43a1-9ad6-cf05b40e520d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:27.666720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6b815-cd16-4718-9c10-5a65f6d0c73d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:27.667090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6b815-cd16-4718-9c10-5a65f6d0c73d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:27.668583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6b815-cd16-4718-9c10-5a65f6d0c73d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:27.673903 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de66d33-a433-43a1-9ad6-cf05b40e520d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:27.675318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de66d33-a433-43a1-9ad6-cf05b40e520d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:27.678646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:27.683607 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 300/590] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-4de66d33-a433-43a1-9ad6-cf05b40e520d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:27.687741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:27.688728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 291/591] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:27.690253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de66d33-a433-43a1-9ad6-cf05b40e520d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.691368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6b815-cd16-4718-9c10-5a65f6d0c73d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.797481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de66d33-a433-43a1-9ad6-cf05b40e520d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.800238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6b815-cd16-4718-9c10-5a65f6d0c73d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.832302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9529f4b-d1d4-4aa7-b130-4173b5b84fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:27.833115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9529f4b-d1d4-4aa7-b130-4173b5b84fe2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:27.833468 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9529f4b-d1d4-4aa7-b130-4173b5b84fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:27.833813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9529f4b-d1d4-4aa7-b130-4173b5b84fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:27.834558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9529f4b-d1d4-4aa7-b130-4173b5b84fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:27.834932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9529f4b-d1d4-4aa7-b130-4173b5b84fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:27.837015 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9529f4b-d1d4-4aa7-b130-4173b5b84fe2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:27.837642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb83c9-3d90-480b-93cf-f84ca9e194f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:27.838271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb83c9-3d90-480b-93cf-f84ca9e194f7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:27.838558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb83c9-3d90-480b-93cf-f84ca9e194f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:27.838834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb83c9-3d90-480b-93cf-f84ca9e194f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:27.854611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 292/592] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:27.854774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb83c9-3d90-480b-93cf-f84ca9e194f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:27.854909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9529f4b-d1d4-4aa7-b130-4173b5b84fe2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb83c9-3d90-480b-93cf-f84ca9e194f7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-04fb83c9-3d90-480b-93cf-f84ca9e194f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:27.869567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:27.870732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:27.871786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:27.871786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:27.871786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:27.872301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 301/593] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:27.877507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb83c9-3d90-480b-93cf-f84ca9e194f7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:27.880464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:27.938301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:27.973701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9529f4b-d1d4-4aa7-b130-4173b5b84fe2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:27.992703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb83c9-3d90-480b-93cf-f84ca9e194f7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:28.009846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aefd72-2267-4b5d-837f-f16273d6ba73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:28.010502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aefd72-2267-4b5d-837f-f16273d6ba73 None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.010780 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aefd72-2267-4b5d-837f-f16273d6ba73 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.011051 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aefd72-2267-4b5d-837f-f16273d6ba73 None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/1b29c5745a8e42ff8232e5ca22b4a4ad/roles/72ffa7e7e6374ac395b22fe6f5723045` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.026266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aefd72-2267-4b5d-837f-f16273d6ba73 None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=1b29c5745a8e42ff8232e5ca22b4a4ad, role_id=72ffa7e7e6374ac395b22fe6f5723045)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:28.027712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aefd72-2267-4b5d-837f-f16273d6ba73 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:28.030123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046d92b-8033-4653-9ef7-14fa3db14d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:28.030936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046d92b-8033-4653-9ef7-14fa3db14d92 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.031405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046d92b-8033-4653-9ef7-14fa3db14d92 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.031677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046d92b-8033-4653-9ef7-14fa3db14d92 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.032404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046d92b-8033-4653-9ef7-14fa3db14d92 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:28.032712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046d92b-8033-4653-9ef7-14fa3db14d92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:28.034142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046d92b-8033-4653-9ef7-14fa3db14d92 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:28.051905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 302/594] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 8 18:33:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:28.052907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046d92b-8033-4653-9ef7-14fa3db14d92 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:28.097919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 293/595] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:33:27 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/1b29c5745a8e42ff8232e5ca22b4a4ad/roles/72ffa7e7e6374ac395b22fe6f5723045 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:28.099180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aefd72-2267-4b5d-837f-f16273d6ba73 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:28.186281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046d92b-8033-4653-9ef7-14fa3db14d92 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:28.197027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:28.201001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:28.228341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbabe6-e76e-475b-b13d-fe8a26b5d55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:28.229269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbabe6-e76e-475b-b13d-fe8a26b5d55e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.229658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbabe6-e76e-475b-b13d-fe8a26b5d55e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.230023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbabe6-e76e-475b-b13d-fe8a26b5d55e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.239123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbabe6-e76e-475b-b13d-fe8a26b5d55e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:28.241524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbabe6-e76e-475b-b13d-fe8a26b5d55e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:28.243976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aefd72-2267-4b5d-837f-f16273d6ba73 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-f3fbabe6-e76e-475b-b13d-fe8a26b5d55e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:28.254560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:28.255692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:28.256770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:28.256770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:28.256770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:28.256770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:28.256770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:28.256770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:28.256770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:28.256770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:28.263381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 303/596] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:28.264689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbabe6-e76e-475b-b13d-fe8a26b5d55e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:28.298944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0a07d-ed29-4444-9240-a4a5c201c71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:28.300358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0a07d-ed29-4444-9240-a4a5c201c71a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.300358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0a07d-ed29-4444-9240-a4a5c201c71a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.300802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0a07d-ed29-4444-9240-a4a5c201c71a None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.301662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0a07d-ed29-4444-9240-a4a5c201c71a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:28.302024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0a07d-ed29-4444-9240-a4a5c201c71a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:28.303372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0a07d-ed29-4444-9240-a4a5c201c71a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:28.316823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 294/597] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:28.318003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0a07d-ed29-4444-9240-a4a5c201c71a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:28.391977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbabe6-e76e-475b-b13d-fe8a26b5d55e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:28.431154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c173d-6ef6-40df-9e82-42fa7ab39bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:28.431953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c173d-6ef6-40df-9e82-42fa7ab39bb5 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.432359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c173d-6ef6-40df-9e82-42fa7ab39bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.432738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c173d-6ef6-40df-9e82-42fa7ab39bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.446209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c173d-6ef6-40df-9e82-42fa7ab39bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:28.446426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0a07d-ed29-4444-9240-a4a5c201c71a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:28.448088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c173d-6ef6-40df-9e82-42fa7ab39bb5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-8d0c173d-6ef6-40df-9e82-42fa7ab39bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:28.466445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:28.467464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:28.468620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:28.468620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:28.468620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:28.473104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 304/598] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:28.474149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c173d-6ef6-40df-9e82-42fa7ab39bb5 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:28.482181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:28.488086 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c0238-4c71-4de3-a025-197064d00a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:28.488722 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c0238-4c71-4de3-a025-197064d00a0c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.489138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c0238-4c71-4de3-a025-197064d00a0c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.496779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c0238-4c71-4de3-a025-197064d00a0c None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.496779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c0238-4c71-4de3-a025-197064d00a0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:28.496779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c0238-4c71-4de3-a025-197064d00a0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:28.496779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c0238-4c71-4de3-a025-197064d00a0c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:28.504464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:28.506091 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 295/599] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:28.507591 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c0238-4c71-4de3-a025-197064d00a0c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:28.586953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c173d-6ef6-40df-9e82-42fa7ab39bb5 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:28.626800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c0238-4c71-4de3-a025-197064d00a0c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:28.628231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac678ad5-e293-465a-b297-ba9100d1b0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:28.628860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac678ad5-e293-465a-b297-ba9100d1b0ad None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.629159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac678ad5-e293-465a-b297-ba9100d1b0ad None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.629432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac678ad5-e293-465a-b297-ba9100d1b0ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.636898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac678ad5-e293-465a-b297-ba9100d1b0ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:28.638375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac678ad5-e293-465a-b297-ba9100d1b0ad None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-ac678ad5-e293-465a-b297-ba9100d1b0ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:28.650524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:28.651433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:28.652358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:28.652358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:28.652358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:28.652358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:28.652358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:28.652358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:28.652358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:28.652358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:28.652358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 305/600] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:28.652358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac678ad5-e293-465a-b297-ba9100d1b0ad None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:28.671962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f38b8d-55c3-486c-821c-8ccee86cae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:28.672879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f38b8d-55c3-486c-821c-8ccee86cae50 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.673209 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f38b8d-55c3-486c-821c-8ccee86cae50 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.673492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f38b8d-55c3-486c-821c-8ccee86cae50 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.674145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f38b8d-55c3-486c-821c-8ccee86cae50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:28.674452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f38b8d-55c3-486c-821c-8ccee86cae50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:28.676279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f38b8d-55c3-486c-821c-8ccee86cae50 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:28.700709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 296/601] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:28.700709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38f53-e12f-45a0-88ec-18adbd20192f None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.700709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38f53-e12f-45a0-88ec-18adbd20192f None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.700709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38f53-e12f-45a0-88ec-18adbd20192f None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.700709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 297/602] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:28.700709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f38f53-e12f-45a0-88ec-18adbd20192f None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:28.739961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:28.757396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:28.758880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac678ad5-e293-465a-b297-ba9100d1b0ad None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:28.794621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57bcc3-7a20-48f4-ace5-6bf36ca0d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:28.795282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57bcc3-7a20-48f4-ace5-6bf36ca0d1d4 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.795564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57bcc3-7a20-48f4-ace5-6bf36ca0d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.795836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57bcc3-7a20-48f4-ace5-6bf36ca0d1d4 None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/605b5d4c0fc9409a94dad22c77c67332/roles/72ffa7e7e6374ac395b22fe6f5723045` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.811572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57bcc3-7a20-48f4-ace5-6bf36ca0d1d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=605b5d4c0fc9409a94dad22c77c67332, role_id=72ffa7e7e6374ac395b22fe6f5723045)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:28.812426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f38f53-e12f-45a0-88ec-18adbd20192f None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:28.813017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57bcc3-7a20-48f4-ace5-6bf36ca0d1d4 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:28.846818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60b9f3-64c0-4198-9c1d-a437cce711ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:28.847352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60b9f3-64c0-4198-9c1d-a437cce711ad None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.847631 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60b9f3-64c0-4198-9c1d-a437cce711ad None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.847904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60b9f3-64c0-4198-9c1d-a437cce711ad None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.848627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60b9f3-64c0-4198-9c1d-a437cce711ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:28.848936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60b9f3-64c0-4198-9c1d-a437cce711ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:28.850408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60b9f3-64c0-4198-9c1d-a437cce711ad None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:28.862653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 298/603] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:28.863894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486285c2-b2d4-4835-a8bf-28016d3875e7 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:28.864180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486285c2-b2d4-4835-a8bf-28016d3875e7 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:28.864459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486285c2-b2d4-4835-a8bf-28016d3875e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:28.884312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-486285c2-b2d4-4835-a8bf-28016d3875e7 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:28.892759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 306/604] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:33:28 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/605b5d4c0fc9409a94dad22c77c67332/roles/72ffa7e7e6374ac395b22fe6f5723045 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:28.892993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-486285c2-b2d4-4835-a8bf-28016d3875e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:28.893121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57bcc3-7a20-48f4-ace5-6bf36ca0d1d4 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:28.973283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 299/605] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:28.975191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486285c2-b2d4-4835-a8bf-28016d3875e7 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:29.064246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57bcc3-7a20-48f4-ace5-6bf36ca0d1d4 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:29.066418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:29.119399 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:29.127797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce862ba-f297-45f8-bdff-ded2218f3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:29.128715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce862ba-f297-45f8-bdff-ded2218f3f2e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:29.129224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce862ba-f297-45f8-bdff-ded2218f3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:29.129732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce862ba-f297-45f8-bdff-ded2218f3f2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:29.140160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce862ba-f297-45f8-bdff-ded2218f3f2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:29.142775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce862ba-f297-45f8-bdff-ded2218f3f2e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-bce862ba-f297-45f8-bdff-ded2218f3f2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:29.158319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:29.164522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:29.165663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:29.165663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:29.165663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:29.165663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:29.165663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:29.165663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:29.165663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:29.165663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:29.165663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 307/606] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:29.166262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486285c2-b2d4-4835-a8bf-28016d3875e7 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:29.169008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce862ba-f297-45f8-bdff-ded2218f3f2e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:29.199493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21334a40-6843-47c3-862c-249628200666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:29.200155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21334a40-6843-47c3-862c-249628200666 None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:29.200431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21334a40-6843-47c3-862c-249628200666 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:29.200702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21334a40-6843-47c3-862c-249628200666 None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/c8d10585480f41608bc6935e282f7e27/roles/72ffa7e7e6374ac395b22fe6f5723045` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:29.215352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21334a40-6843-47c3-862c-249628200666 None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=c8d10585480f41608bc6935e282f7e27, role_id=72ffa7e7e6374ac395b22fe6f5723045)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:29.216788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21334a40-6843-47c3-862c-249628200666 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:29.297507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce862ba-f297-45f8-bdff-ded2218f3f2e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:29.311415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 300/607] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:33:28 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/c8d10585480f41608bc6935e282f7e27/roles/72ffa7e7e6374ac395b22fe6f5723045 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:29.312076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21334a40-6843-47c3-862c-249628200666 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:29.357852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84cb3b-72d6-427c-9a9d-460506961ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:29.359822 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84cb3b-72d6-427c-9a9d-460506961ef2 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:29.360552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84cb3b-72d6-427c-9a9d-460506961ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:29.360910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84cb3b-72d6-427c-9a9d-460506961ef2 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:29.362990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84cb3b-72d6-427c-9a9d-460506961ef2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:29.364513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84cb3b-72d6-427c-9a9d-460506961ef2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:29.366144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84cb3b-72d6-427c-9a9d-460506961ef2 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:29.390418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 308/608] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:29.396807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84cb3b-72d6-427c-9a9d-460506961ef2 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:29.513493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:29.522916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:29.539065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21334a40-6843-47c3-862c-249628200666 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:29.566196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84cb3b-72d6-427c-9a9d-460506961ef2 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:29.597124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fd4f0-b7c0-4cdf-9119-0a136499b404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:29.598705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fd4f0-b7c0-4cdf-9119-0a136499b404 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:29.599143 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fd4f0-b7c0-4cdf-9119-0a136499b404 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:29.599568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fd4f0-b7c0-4cdf-9119-0a136499b404 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:29.613176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c14cc-a10b-4a7b-96b3-9bc936f6225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:29.614540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c14cc-a10b-4a7b-96b3-9bc936f6225d None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:29.614540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c14cc-a10b-4a7b-96b3-9bc936f6225d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:29.614830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c14cc-a10b-4a7b-96b3-9bc936f6225d None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/bcf30ef695154d3a97d7e589aca21233/roles/72ffa7e7e6374ac395b22fe6f5723045` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:29.616971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fd4f0-b7c0-4cdf-9119-0a136499b404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-3c3fd4f0-b7c0-4cdf-9119-0a136499b404 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:29.623850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:29.625791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:29.627092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:29.630176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 301/609] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:29.637491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fd4f0-b7c0-4cdf-9119-0a136499b404 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:29.649002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c14cc-a10b-4a7b-96b3-9bc936f6225d None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=bcf30ef695154d3a97d7e589aca21233, role_id=72ffa7e7e6374ac395b22fe6f5723045)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:29.649002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c14cc-a10b-4a7b-96b3-9bc936f6225d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:29.724472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 309/610] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:33:29 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/bcf30ef695154d3a97d7e589aca21233/roles/72ffa7e7e6374ac395b22fe6f5723045 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:29.775195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fd4f0-b7c0-4cdf-9119-0a136499b404 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:29.818381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9248e81-daaf-47e5-9cbf-ac7727bfd77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:29.819163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9248e81-daaf-47e5-9cbf-ac7727bfd77a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:29.820278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9248e81-daaf-47e5-9cbf-ac7727bfd77a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:29.820698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9248e81-daaf-47e5-9cbf-ac7727bfd77a None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:29.821554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9248e81-daaf-47e5-9cbf-ac7727bfd77a None admin] RBAC: Adding query filter params (name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:29.821962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9248e81-daaf-47e5-9cbf-ac7727bfd77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:29.823507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9248e81-daaf-47e5-9cbf-ac7727bfd77a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:29.838815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 302/611] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:29.844599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c14cc-a10b-4a7b-96b3-9bc936f6225d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:29.993866 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c14cc-a10b-4a7b-96b3-9bc936f6225d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:29.995677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:30.038407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32097c8-6350-4ad5-9a6e-92457015e233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:30.039328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32097c8-6350-4ad5-9a6e-92457015e233 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:30.039663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32097c8-6350-4ad5-9a6e-92457015e233 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:30.039944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32097c8-6350-4ad5-9a6e-92457015e233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:30.051433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32097c8-6350-4ad5-9a6e-92457015e233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:30.052892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32097c8-6350-4ad5-9a6e-92457015e233 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-c32097c8-6350-4ad5-9a6e-92457015e233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:30.075539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:30.076915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:30.078216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:30.078216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:30.078216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:30.078216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 310/612] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:30.083769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9248e81-daaf-47e5-9cbf-ac7727bfd77a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:30.114944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:30.236820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9248e81-daaf-47e5-9cbf-ac7727bfd77a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:30.286959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391b50f-a46f-46e2-ade6-a5272310401f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:30.287914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391b50f-a46f-46e2-ade6-a5272310401f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:30.288927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391b50f-a46f-46e2-ade6-a5272310401f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:30.289451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391b50f-a46f-46e2-ade6-a5272310401f None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:30.290322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391b50f-a46f-46e2-ade6-a5272310401f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:30.290844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391b50f-a46f-46e2-ade6-a5272310401f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:30.292429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391b50f-a46f-46e2-ade6-a5272310401f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:30.308432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 303/613] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:30.313530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32097c8-6350-4ad5-9a6e-92457015e233 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:30.484252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32097c8-6350-4ad5-9a6e-92457015e233 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:30.538145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a13266-306d-47bf-bcc0-fc9fe50d6385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:30.541316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13266-306d-47bf-bcc0-fc9fe50d6385 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:30.541316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13266-306d-47bf-bcc0-fc9fe50d6385 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:30.541316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a13266-306d-47bf-bcc0-fc9fe50d6385 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:30.553002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a13266-306d-47bf-bcc0-fc9fe50d6385 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:30.555305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a13266-306d-47bf-bcc0-fc9fe50d6385 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-e4a13266-306d-47bf-bcc0-fc9fe50d6385 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:30.568633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:30.570778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:30.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:30.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:30.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:30.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:30.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:30.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:30.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:30.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:30.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:30.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:30.578349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 311/614] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:30.579625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391b50f-a46f-46e2-ade6-a5272310401f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:30.608163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32fa45-80d5-422b-9f21-ecd557d8d976 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:30.608667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32fa45-80d5-422b-9f21-ecd557d8d976 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:30.609160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32fa45-80d5-422b-9f21-ecd557d8d976 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:30.610695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 312/615] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:33:30.615466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1ade6-8136-4d5a-b3fc-6c4012d19d31 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:30.615992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1ade6-8136-4d5a-b3fc-6c4012d19d31 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:30.616978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1ade6-8136-4d5a-b3fc-6c4012d19d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:30.644537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-62e1ade6-8136-4d5a-b3fc-6c4012d19d31 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:30.646703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-62e1ade6-8136-4d5a-b3fc-6c4012d19d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:30.667306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:30.728496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:30.777687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 313/616] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:33:30.782796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391b50f-a46f-46e2-ade6-a5272310401f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:30.826829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42752ad-8c67-4d32-b84f-d5137fb81e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:30.827545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42752ad-8c67-4d32-b84f-d5137fb81e7d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:30.827926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42752ad-8c67-4d32-b84f-d5137fb81e7d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:30.828300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42752ad-8c67-4d32-b84f-d5137fb81e7d None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:30.829160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42752ad-8c67-4d32-b84f-d5137fb81e7d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:30.829574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42752ad-8c67-4d32-b84f-d5137fb81e7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:30.831104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42752ad-8c67-4d32-b84f-d5137fb81e7d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:30.859105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 304/617] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 18:33:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:30.864807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1ade6-8136-4d5a-b3fc-6c4012d19d31 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:31.001133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1ade6-8136-4d5a-b3fc-6c4012d19d31 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:31.037966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ed07e-0a90-4c74-aa79-e6a230a81a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:31.039086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ed07e-0a90-4c74-aa79-e6a230a81a6d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:31.039544 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ed07e-0a90-4c74-aa79-e6a230a81a6d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:31.039977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ed07e-0a90-4c74-aa79-e6a230a81a6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:31.055714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ed07e-0a90-4c74-aa79-e6a230a81a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:31.057410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ed07e-0a90-4c74-aa79-e6a230a81a6d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:31.060334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-4f8ed07e-0a90-4c74-aa79-e6a230a81a6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:31.086584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:31.087971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:31.089354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:31.089354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:31.089354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:31.089354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 314/618] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:31.089640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42752ad-8c67-4d32-b84f-d5137fb81e7d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:31.089728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ed07e-0a90-4c74-aa79-e6a230a81a6d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:31.204804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42752ad-8c67-4d32-b84f-d5137fb81e7d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:31.232970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ed07e-0a90-4c74-aa79-e6a230a81a6d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:31.240953 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:31.248210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345a57-0901-47cb-9ac7-d4aba3ae3a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:31.248955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345a57-0901-47cb-9ac7-d4aba3ae3a23 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:31.253202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345a57-0901-47cb-9ac7-d4aba3ae3a23 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:31.253202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345a57-0901-47cb-9ac7-d4aba3ae3a23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:31.263540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4345a57-0901-47cb-9ac7-d4aba3ae3a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-e4345a57-0901-47cb-9ac7-d4aba3ae3a23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:31.269914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:31.270851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:31.271747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:31.275216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 305/619] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:31.281558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345a57-0901-47cb-9ac7-d4aba3ae3a23 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:31.318295 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888b4cf-3d6f-4d20-ae1a-cd078a3d28dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:31.319326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888b4cf-3d6f-4d20-ae1a-cd078a3d28dc None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:31.319869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888b4cf-3d6f-4d20-ae1a-cd078a3d28dc None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:31.320430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888b4cf-3d6f-4d20-ae1a-cd078a3d28dc None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:31.321339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888b4cf-3d6f-4d20-ae1a-cd078a3d28dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:31.321851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888b4cf-3d6f-4d20-ae1a-cd078a3d28dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:31.323449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888b4cf-3d6f-4d20-ae1a-cd078a3d28dc None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:31.351971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 315/620] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:31.357415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888b4cf-3d6f-4d20-ae1a-cd078a3d28dc None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:31.427525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:31.487123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888b4cf-3d6f-4d20-ae1a-cd078a3d28dc None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:31.515258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345a57-0901-47cb-9ac7-d4aba3ae3a23 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:31.537594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc392a-62be-4e44-a9ba-acee72156827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:31.539807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc392a-62be-4e44-a9ba-acee72156827 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:31.540095 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc392a-62be-4e44-a9ba-acee72156827 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:31.540370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc392a-62be-4e44-a9ba-acee72156827 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:31.553397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc392a-62be-4e44-a9ba-acee72156827 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:31.554917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc392a-62be-4e44-a9ba-acee72156827 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-98fc392a-62be-4e44-a9ba-acee72156827 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:31.575401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:31.576375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:31.578189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:31.578189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:31.578189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:31.578189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:31.578189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:31.578189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:31.578189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:31.578189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:31.578189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 316/621] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:31.592130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc392a-62be-4e44-a9ba-acee72156827 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:31.609942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ea9e6-6b9e-458a-a958-9edee93c8229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:31.609942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ea9e6-6b9e-458a-a958-9edee93c8229 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:31.609942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ea9e6-6b9e-458a-a958-9edee93c8229 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:31.609942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ea9e6-6b9e-458a-a958-9edee93c8229 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:31.610620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ea9e6-6b9e-458a-a958-9edee93c8229 None admin] RBAC: Adding query filter params (name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:31.610690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ea9e6-6b9e-458a-a958-9edee93c8229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:31.613823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ea9e6-6b9e-458a-a958-9edee93c8229 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:31.623926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 306/622] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:31.630013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ea9e6-6b9e-458a-a958-9edee93c8229 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:31.722014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:31.815893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ea9e6-6b9e-458a-a958-9edee93c8229 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:31.818036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc392a-62be-4e44-a9ba-acee72156827 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:31.850356 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:31.883539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d04a81-7335-47da-8f41-f73309a6fbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:31.883747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a24e9-336a-4784-ac15-840da549b3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:31.884153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a24e9-336a-4784-ac15-840da549b3b0 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:31.884434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a24e9-336a-4784-ac15-840da549b3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:31.884706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a24e9-336a-4784-ac15-840da549b3b0 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:31.885604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a24e9-336a-4784-ac15-840da549b3b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:31.885830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a24e9-336a-4784-ac15-840da549b3b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:31.886082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d04a81-7335-47da-8f41-f73309a6fbde None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:31.886555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d04a81-7335-47da-8f41-f73309a6fbde None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:31.887018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d04a81-7335-47da-8f41-f73309a6fbde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:31.887304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a24e9-336a-4784-ac15-840da549b3b0 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:31.904459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 317/623] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:31.905471 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d04a81-7335-47da-8f41-f73309a6fbde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:31.905863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6d39c-1957-4321-95e6-895a4311dfc2 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:31.906150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6d39c-1957-4321-95e6-895a4311dfc2 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:31.906425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6d39c-1957-4321-95e6-895a4311dfc2 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:31.912118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 318/624] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:31.912118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6d39c-1957-4321-95e6-895a4311dfc2 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:31.912657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d04a81-7335-47da-8f41-f73309a6fbde None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-d5d04a81-7335-47da-8f41-f73309a6fbde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:31.929198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:31.930416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:31.931339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:31.931339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:31.931339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:31.938290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 307/625] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:31.940105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c54043-5cec-4a6c-9b41-7174008aa49b None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:31.947293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c54043-5cec-4a6c-9b41-7174008aa49b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:31.947293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c54043-5cec-4a6c-9b41-7174008aa49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:31.967420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-19c54043-5cec-4a6c-9b41-7174008aa49b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:31.969377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-19c54043-5cec-4a6c-9b41-7174008aa49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:32.030498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6d39c-1957-4321-95e6-895a4311dfc2 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:32.049337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 308/626] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:32.050403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c54043-5cec-4a6c-9b41-7174008aa49b None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:32.070568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de417eb3-f4d0-414b-a10f-a2acf66554bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:32.071492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417eb3-f4d0-414b-a10f-a2acf66554bc None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.071926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417eb3-f4d0-414b-a10f-a2acf66554bc None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.072372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417eb3-f4d0-414b-a10f-a2acf66554bc None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/88a9b2cee21f4831b221d113ddbbd29d/roles/72ffa7e7e6374ac395b22fe6f5723045` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.090845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de417eb3-f4d0-414b-a10f-a2acf66554bc None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=88a9b2cee21f4831b221d113ddbbd29d, role_id=72ffa7e7e6374ac395b22fe6f5723045)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:32.092864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de417eb3-f4d0-414b-a10f-a2acf66554bc None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:32.096600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:32.167884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c54043-5cec-4a6c-9b41-7174008aa49b None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:32.171856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 319/627] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:33:31 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/88a9b2cee21f4831b221d113ddbbd29d/roles/72ffa7e7e6374ac395b22fe6f5723045 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:32.173392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fcae5-3cfa-467b-9e12-121804e8fc5e None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.173839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fcae5-3cfa-467b-9e12-121804e8fc5e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.174253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fcae5-3cfa-467b-9e12-121804e8fc5e None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.175668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 320/628] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:32.183416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf9f91-5884-4e6c-a5f9-63a1e81d45d6 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.183978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf9f91-5884-4e6c-a5f9-63a1e81d45d6 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.184431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf9f91-5884-4e6c-a5f9-63a1e81d45d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.223170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-9abf9f91-5884-4e6c-a5f9-63a1e81d45d6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:32.226609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9abf9f91-5884-4e6c-a5f9-63a1e81d45d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:32.258961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:32.259508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427e674-6231-44ff-8947-c63ca7713ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:32.260148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427e674-6231-44ff-8947-c63ca7713ce5 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.260435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427e674-6231-44ff-8947-c63ca7713ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.261075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427e674-6231-44ff-8947-c63ca7713ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.263174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427e674-6231-44ff-8947-c63ca7713ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:32.263174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427e674-6231-44ff-8947-c63ca7713ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:32.264348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427e674-6231-44ff-8947-c63ca7713ce5 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:32.287133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 309/629] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:32.293648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427e674-6231-44ff-8947-c63ca7713ce5 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:32.356621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 321/630] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:33:32.358252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf9f91-5884-4e6c-a5f9-63a1e81d45d6 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:32.454731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:32.467083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427e674-6231-44ff-8947-c63ca7713ce5 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:32.505613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf9f91-5884-4e6c-a5f9-63a1e81d45d6 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:32.506485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7db72a-07fb-4718-ab59-e2cb612e1c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:32.507778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7db72a-07fb-4718-ab59-e2cb612e1c51 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.508060 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7db72a-07fb-4718-ab59-e2cb612e1c51 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.508332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7db72a-07fb-4718-ab59-e2cb612e1c51 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.516382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7db72a-07fb-4718-ab59-e2cb612e1c51 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:32.517880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7db72a-07fb-4718-ab59-e2cb612e1c51 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-4a7db72a-07fb-4718-ab59-e2cb612e1c51 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:32.531168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:32.532197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:32.533246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 310/631] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 18:33:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:32.534076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff9def-feef-4f40-9085-cc7f161554bd None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.534203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff9def-feef-4f40-9085-cc7f161554bd None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.534488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff9def-feef-4f40-9085-cc7f161554bd None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.536032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 311/632] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:32.537644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff9def-feef-4f40-9085-cc7f161554bd None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:32.558161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6cdd6-14fa-444b-9c07-cc129ee0ad0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:32.559075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6cdd6-14fa-444b-9c07-cc129ee0ad0f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.559670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6cdd6-14fa-444b-9c07-cc129ee0ad0f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.560099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6cdd6-14fa-444b-9c07-cc129ee0ad0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.577552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6cdd6-14fa-444b-9c07-cc129ee0ad0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-30d6cdd6-14fa-444b-9c07-cc129ee0ad0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:32.582929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:32.584072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:32.585194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:32.588977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 322/633] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:32.590445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2d416-84c1-48a9-99fc-9a912459cb80 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.590886 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2d416-84c1-48a9-99fc-9a912459cb80 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.591301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2d416-84c1-48a9-99fc-9a912459cb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.614384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8ee2d416-84c1-48a9-99fc-9a912459cb80 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:32.615836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8ee2d416-84c1-48a9-99fc-9a912459cb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:32.633428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:32.665116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff9def-feef-4f40-9085-cc7f161554bd None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:32.690495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 323/634] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:32.691348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2d416-84c1-48a9-99fc-9a912459cb80 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:32.713677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff966102-9a0f-43ad-8987-12f9befb4a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:32.714595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff966102-9a0f-43ad-8987-12f9befb4a2c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.715532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff966102-9a0f-43ad-8987-12f9befb4a2c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.716008 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff966102-9a0f-43ad-8987-12f9befb4a2c None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.717286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff966102-9a0f-43ad-8987-12f9befb4a2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:32.717951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff966102-9a0f-43ad-8987-12f9befb4a2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:32.720590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff966102-9a0f-43ad-8987-12f9befb4a2c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:32.744258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 312/635] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 18:33:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:32.745592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff966102-9a0f-43ad-8987-12f9befb4a2c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:32.764275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:32.803397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2d416-84c1-48a9-99fc-9a912459cb80 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:32.841897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adea63-685f-4927-95c2-f8daca9c4514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:32.842555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adea63-685f-4927-95c2-f8daca9c4514 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.842811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adea63-685f-4927-95c2-f8daca9c4514 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.849072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99adea63-685f-4927-95c2-f8daca9c4514 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.849072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adea63-685f-4927-95c2-f8daca9c4514 None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:32.849072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adea63-685f-4927-95c2-f8daca9c4514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:32.849072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99adea63-685f-4927-95c2-f8daca9c4514 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:32.855593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 324/636] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:32.856459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adea63-685f-4927-95c2-f8daca9c4514 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:32.860837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff966102-9a0f-43ad-8987-12f9befb4a2c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:32.868606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:32.896994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf80680-dc55-4910-b9a9-539d140d7f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:32.898517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf80680-dc55-4910-b9a9-539d140d7f4f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:32.898844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf80680-dc55-4910-b9a9-539d140d7f4f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:32.899122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf80680-dc55-4910-b9a9-539d140d7f4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:32.910344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf80680-dc55-4910-b9a9-539d140d7f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-bdf80680-dc55-4910-b9a9-539d140d7f4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:32.915498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:32.916612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:32.917650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 313/637] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:32.918372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf80680-dc55-4910-b9a9-539d140d7f4f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:32.975158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99adea63-685f-4927-95c2-f8daca9c4514 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:32.993136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:33.014453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6007704-f69e-42bd-b723-f5595ce8fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.015140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6007704-f69e-42bd-b723-f5595ce8fa1a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.015388 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6007704-f69e-42bd-b723-f5595ce8fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.015660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6007704-f69e-42bd-b723-f5595ce8fa1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.028077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf80680-dc55-4910-b9a9-539d140d7f4f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:33.065991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f2ade-b729-4938-a828-c8730e9f8347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.066629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f2ade-b729-4938-a828-c8730e9f8347 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.066913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f2ade-b729-4938-a828-c8730e9f8347 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.067190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f2ade-b729-4938-a828-c8730e9f8347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.079955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f2ade-b729-4938-a828-c8730e9f8347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:33.081702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f2ade-b729-4938-a828-c8730e9f8347 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:33.091776 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6007704-f69e-42bd-b723-f5595ce8fa1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:33.093541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6007704-f69e-42bd-b723-f5595ce8fa1a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-cb6f2ade-b729-4938-a828-c8730e9f8347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:33.097476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:33.098436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:33.099330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:33.099330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:33.099330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:33.099330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 314/638] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:33.100118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f2ade-b729-4938-a828-c8730e9f8347 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.108690 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:33.201005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f2ade-b729-4938-a828-c8730e9f8347 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:33.211762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 325/639] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:33.211762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6007704-f69e-42bd-b723-f5595ce8fa1a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.230977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:33.242223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0383d6e-2cd0-4eee-b8a9-5b521a46ab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.242837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0383d6e-2cd0-4eee-b8a9-5b521a46ab56 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.243117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0383d6e-2cd0-4eee-b8a9-5b521a46ab56 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.243392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0383d6e-2cd0-4eee-b8a9-5b521a46ab56 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.244065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0383d6e-2cd0-4eee-b8a9-5b521a46ab56 None admin] RBAC: Adding query filter params (name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:33.244370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0383d6e-2cd0-4eee-b8a9-5b521a46ab56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:33.245762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0383d6e-2cd0-4eee-b8a9-5b521a46ab56 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:33.254714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 315/640] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:33.255515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0383d6e-2cd0-4eee-b8a9-5b521a46ab56 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.324835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6007704-f69e-42bd-b723-f5595ce8fa1a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:33.335473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:33.356713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0383d6e-2cd0-4eee-b8a9-5b521a46ab56 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:33.362312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c48788-81fc-4b00-8e8c-f09d515a3ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.362947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c48788-81fc-4b00-8e8c-f09d515a3ad5 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.363229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c48788-81fc-4b00-8e8c-f09d515a3ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.363560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c48788-81fc-4b00-8e8c-f09d515a3ad5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.375834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c48788-81fc-4b00-8e8c-f09d515a3ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-e4c48788-81fc-4b00-8e8c-f09d515a3ad5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:33.381082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:33.382263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 326/641] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:33.388170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c48788-81fc-4b00-8e8c-f09d515a3ad5 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.400603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e87c0-6207-446b-ab18-41b038d76463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.401245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e87c0-6207-446b-ab18-41b038d76463 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.401521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e87c0-6207-446b-ab18-41b038d76463 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.401791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e87c0-6207-446b-ab18-41b038d76463 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.402434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e87c0-6207-446b-ab18-41b038d76463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:33.402738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e87c0-6207-446b-ab18-41b038d76463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:33.404159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e87c0-6207-446b-ab18-41b038d76463 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:33.420043 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 316/642] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:33.420043 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e87c0-6207-446b-ab18-41b038d76463 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.464932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:33.501749 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c48788-81fc-4b00-8e8c-f09d515a3ad5 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:33.523186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e87c0-6207-446b-ab18-41b038d76463 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:33.542002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc63f49-8ade-4f3d-89fb-420288a58a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.542770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc63f49-8ade-4f3d-89fb-420288a58a30 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.543174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc63f49-8ade-4f3d-89fb-420288a58a30 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.543541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc63f49-8ade-4f3d-89fb-420288a58a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.555782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc63f49-8ade-4f3d-89fb-420288a58a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:33.557378 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc63f49-8ade-4f3d-89fb-420288a58a30 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:33.558960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014fc23-70d2-4c6a-8a7c-350abb40b2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.559662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014fc23-70d2-4c6a-8a7c-350abb40b2a8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.560010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014fc23-70d2-4c6a-8a7c-350abb40b2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.560352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014fc23-70d2-4c6a-8a7c-350abb40b2a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.571462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014fc23-70d2-4c6a-8a7c-350abb40b2a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-4bc63f49-8ade-4f3d-89fb-420288a58a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:33.572148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:33.573389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:33.574521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:33.574521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:33.574521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:33.574736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014fc23-70d2-4c6a-8a7c-350abb40b2a8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:33.579769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 327/643] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:33.579769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc63f49-8ade-4f3d-89fb-420288a58a30 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-e014fc23-70d2-4c6a-8a7c-350abb40b2a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:33.599643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:33.601474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:33.602552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:33.602552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:33.602552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:33.602552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 317/644] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:33.607343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014fc23-70d2-4c6a-8a7c-350abb40b2a8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.607710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:33.692089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc63f49-8ade-4f3d-89fb-420288a58a30 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:33.709343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:33.713777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014fc23-70d2-4c6a-8a7c-350abb40b2a8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:33.728367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e797849-6c34-4672-851e-77a9cfffbca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.729936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e797849-6c34-4672-851e-77a9cfffbca5 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.729936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e797849-6c34-4672-851e-77a9cfffbca5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.729936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e797849-6c34-4672-851e-77a9cfffbca5 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.730494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e797849-6c34-4672-851e-77a9cfffbca5 None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:33.730863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e797849-6c34-4672-851e-77a9cfffbca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:33.732286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e797849-6c34-4672-851e-77a9cfffbca5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:33.740737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 328/645] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:33.745996 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e797849-6c34-4672-851e-77a9cfffbca5 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.747920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ac9df-32b5-42e4-b77f-63a0f15073e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.748565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac9df-32b5-42e4-b77f-63a0f15073e1 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.749260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac9df-32b5-42e4-b77f-63a0f15073e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.749545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ac9df-32b5-42e4-b77f-63a0f15073e1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.757922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ac9df-32b5-42e4-b77f-63a0f15073e1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:33.759271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ac9df-32b5-42e4-b77f-63a0f15073e1 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-7e1ac9df-32b5-42e4-b77f-63a0f15073e1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:33.770668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:33.776520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 318/646] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:33.777774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ac9df-32b5-42e4-b77f-63a0f15073e1 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.846824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:33.866947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e797849-6c34-4672-851e-77a9cfffbca5 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:33.881117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ac9df-32b5-42e4-b77f-63a0f15073e1 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:33.904826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fe8b2-308a-4f02-9863-4aad5b10680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.905454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fe8b2-308a-4f02-9863-4aad5b10680f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.905734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fe8b2-308a-4f02-9863-4aad5b10680f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.906008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fe8b2-308a-4f02-9863-4aad5b10680f None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.906648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fe8b2-308a-4f02-9863-4aad5b10680f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:33.906950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fe8b2-308a-4f02-9863-4aad5b10680f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:33.908305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fe8b2-308a-4f02-9863-4aad5b10680f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:33.918720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97c109-1e5b-483b-8f05-8690dfe4f684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:33.919373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97c109-1e5b-483b-8f05-8690dfe4f684 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:33.919653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97c109-1e5b-483b-8f05-8690dfe4f684 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:33.919930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97c109-1e5b-483b-8f05-8690dfe4f684 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:33.920565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97c109-1e5b-483b-8f05-8690dfe4f684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:33.925233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97c109-1e5b-483b-8f05-8690dfe4f684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:33.925386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 329/647] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:33.925386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fe8b2-308a-4f02-9863-4aad5b10680f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.925546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97c109-1e5b-483b-8f05-8690dfe4f684 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:33.936801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 319/648] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:33.937660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97c109-1e5b-483b-8f05-8690dfe4f684 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:33.956403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:34.038369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fe8b2-308a-4f02-9863-4aad5b10680f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.046765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97c109-1e5b-483b-8f05-8690dfe4f684 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.075806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878bed8-dd6b-4b63-9d25-5b086dc1e395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.076573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878bed8-dd6b-4b63-9d25-5b086dc1e395 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.076964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878bed8-dd6b-4b63-9d25-5b086dc1e395 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.077367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878bed8-dd6b-4b63-9d25-5b086dc1e395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.085265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aba9da-b40a-4316-80bf-d0324decd4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.085929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aba9da-b40a-4316-80bf-d0324decd4fe None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.086286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aba9da-b40a-4316-80bf-d0324decd4fe None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.087005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aba9da-b40a-4316-80bf-d0324decd4fe None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.088012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aba9da-b40a-4316-80bf-d0324decd4fe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:34.088317 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aba9da-b40a-4316-80bf-d0324decd4fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.088635 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878bed8-dd6b-4b63-9d25-5b086dc1e395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.089777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aba9da-b40a-4316-80bf-d0324decd4fe None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.090377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878bed8-dd6b-4b63-9d25-5b086dc1e395 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.092459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:34.108792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 320/649] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:34.109855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aba9da-b40a-4316-80bf-d0324decd4fe None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-7878bed8-dd6b-4b63-9d25-5b086dc1e395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:34.111079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:34.112054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:34.112944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:34.112944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:34.112944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:34.117375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 330/650] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:34.118485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878bed8-dd6b-4b63-9d25-5b086dc1e395 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:34.193137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:34.218831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aba9da-b40a-4316-80bf-d0324decd4fe None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.232160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878bed8-dd6b-4b63-9d25-5b086dc1e395 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.255503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a221f9-7d01-4885-afe1-ac5af495aafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.256251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a221f9-7d01-4885-afe1-ac5af495aafd None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.256545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a221f9-7d01-4885-afe1-ac5af495aafd None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.256922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a221f9-7d01-4885-afe1-ac5af495aafd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.264450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a221f9-7d01-4885-afe1-ac5af495aafd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.265920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a221f9-7d01-4885-afe1-ac5af495aafd None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.266773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811a168-7020-4308-819b-6b3f744ebcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.267394 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811a168-7020-4308-819b-6b3f744ebcec None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.267677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811a168-7020-4308-819b-6b3f744ebcec None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.267954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811a168-7020-4308-819b-6b3f744ebcec None admin] PATH_INFO: `/v3/users/glance` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.275917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811a168-7020-4308-819b-6b3f744ebcec None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.277361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811a168-7020-4308-819b-6b3f744ebcec None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-f9a221f9-7d01-4885-afe1-ac5af495aafd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:34.278768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:34.279638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:34.280522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:34.280522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:34.280522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:34.280522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:34.280522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:34.280522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:34.280522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:34.280522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:34.282612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 321/651] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:34.283543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a221f9-7d01-4885-afe1-ac5af495aafd None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-5811a168-7020-4308-819b-6b3f744ebcec None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:34.292806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:34.293831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:34.294864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 331/652] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:34.295469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811a168-7020-4308-819b-6b3f744ebcec None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:34.355414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:34.400354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a221f9-7d01-4885-afe1-ac5af495aafd None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.405648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811a168-7020-4308-819b-6b3f744ebcec None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.441307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a2e39-6f78-4ef7-b57c-ef9615d08cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.441927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a2e39-6f78-4ef7-b57c-ef9615d08cd6 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.442209 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a2e39-6f78-4ef7-b57c-ef9615d08cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.442488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a2e39-6f78-4ef7-b57c-ef9615d08cd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.446844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:34.447453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2628746-b6d6-4bde-bea6-37fe5e4d4469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.448061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2628746-b6d6-4bde-bea6-37fe5e4d4469 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.448338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2628746-b6d6-4bde-bea6-37fe5e4d4469 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.448609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2628746-b6d6-4bde-bea6-37fe5e4d4469 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.449280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2628746-b6d6-4bde-bea6-37fe5e4d4469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:34.449584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2628746-b6d6-4bde-bea6-37fe5e4d4469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.450938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2628746-b6d6-4bde-bea6-37fe5e4d4469 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.463909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 332/653] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:34.465967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2628746-b6d6-4bde-bea6-37fe5e4d4469 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:34.519019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a2e39-6f78-4ef7-b57c-ef9615d08cd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.520508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a2e39-6f78-4ef7-b57c-ef9615d08cd6 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.577109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2628746-b6d6-4bde-bea6-37fe5e4d4469 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.591758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:34.618184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2cb23-062b-4766-9608-f0f3b5c8618f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.618801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2cb23-062b-4766-9608-f0f3b5c8618f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.619081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2cb23-062b-4766-9608-f0f3b5c8618f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.619355 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2cb23-062b-4766-9608-f0f3b5c8618f None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.620054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2cb23-062b-4766-9608-f0f3b5c8618f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:34.620363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2cb23-062b-4766-9608-f0f3b5c8618f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.621827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2cb23-062b-4766-9608-f0f3b5c8618f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.622100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 322/654] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:34.622894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a2e39-6f78-4ef7-b57c-ef9615d08cd6 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:34.637328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 333/655] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:34.638141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2cb23-062b-4766-9608-f0f3b5c8618f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:34.657127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:34.721419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a2e39-6f78-4ef7-b57c-ef9615d08cd6 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.738923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2cb23-062b-4766-9608-f0f3b5c8618f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.755914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375935f-e0fa-4b06-a103-13b5da56da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.756510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375935f-e0fa-4b06-a103-13b5da56da69 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.756905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375935f-e0fa-4b06-a103-13b5da56da69 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.757213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375935f-e0fa-4b06-a103-13b5da56da69 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.758111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375935f-e0fa-4b06-a103-13b5da56da69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:34.758752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375935f-e0fa-4b06-a103-13b5da56da69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.759840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375935f-e0fa-4b06-a103-13b5da56da69 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.772039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a662a-5600-447d-95a2-33197de4d0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.772688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a662a-5600-447d-95a2-33197de4d0e7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.772966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a662a-5600-447d-95a2-33197de4d0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.773263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a662a-5600-447d-95a2-33197de4d0e7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.777282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 323/656] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 8 18:33:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:34.778467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa6672-0ca7-4e9b-af1a-bc06cdba89ea None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.778750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa6672-0ca7-4e9b-af1a-bc06cdba89ea None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.779021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa6672-0ca7-4e9b-af1a-bc06cdba89ea None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.780355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 324/657] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:34.780608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a662a-5600-447d-95a2-33197de4d0e7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.780898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa6672-0ca7-4e9b-af1a-bc06cdba89ea None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:34.782058 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a662a-5600-447d-95a2-33197de4d0e7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-833a662a-5600-447d-95a2-33197de4d0e7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:34.793807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:34.794906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 334/658] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:34.795860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a662a-5600-447d-95a2-33197de4d0e7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:34.829982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:34.881419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:34.888910 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa6672-0ca7-4e9b-af1a-bc06cdba89ea None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.896966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a662a-5600-447d-95a2-33197de4d0e7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:34.923225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbba61b-b136-4fe8-b00b-bcb90286ab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.923853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbba61b-b136-4fe8-b00b-bcb90286ab01 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.924135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbba61b-b136-4fe8-b00b-bcb90286ab01 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.924409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbba61b-b136-4fe8-b00b-bcb90286ab01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.931627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e4320-bc0b-4331-922d-2f9b6ef1320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:34.932252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e4320-bc0b-4331-922d-2f9b6ef1320b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.932525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e4320-bc0b-4331-922d-2f9b6ef1320b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.932793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e4320-bc0b-4331-922d-2f9b6ef1320b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.933540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e4320-bc0b-4331-922d-2f9b6ef1320b None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=1b29c5745a8e42ff8232e5ca22b4a4ad) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:34.933843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e4320-bc0b-4331-922d-2f9b6ef1320b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.934976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbba61b-b136-4fe8-b00b-bcb90286ab01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:34.935263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e4320-bc0b-4331-922d-2f9b6ef1320b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.936425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbba61b-b136-4fe8-b00b-bcb90286ab01 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:34.941085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 335/659] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:34 2022] GET /identity/v3/role_assignments?user.id=1b29c5745a8e42ff8232e5ca22b4a4ad&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:34.941724 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e4320-bc0b-4331-922d-2f9b6ef1320b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-bfbba61b-b136-4fe8-b00b-bcb90286ab01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:34.952472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:34.953634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:34.954540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:34.954540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:34.954540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:34.954540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 325/660] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:34.956254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bfa05-dc81-43a3-a9a6-a987cbf5debe None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:34.956254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bfa05-dc81-43a3-a9a6-a987cbf5debe None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:34.956254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bfa05-dc81-43a3-a9a6-a987cbf5debe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:34.978291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2c3bfa05-dc81-43a3-a9a6-a987cbf5debe None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:34.979574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2c3bfa05-dc81-43a3-a9a6-a987cbf5debe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:35.067050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 326/661] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:35.068319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3bfa05-dc81-43a3-a9a6-a987cbf5debe None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:35.081187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e4320-bc0b-4331-922d-2f9b6ef1320b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:35.091693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:35.119082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa463e3a-a87f-470a-8215-b6160930e009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:35.120057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa463e3a-a87f-470a-8215-b6160930e009 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:35.120513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa463e3a-a87f-470a-8215-b6160930e009 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:35.120984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa463e3a-a87f-470a-8215-b6160930e009 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:35.170810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:35.198247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3bfa05-dc81-43a3-a9a6-a987cbf5debe None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:35.211231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa463e3a-a87f-470a-8215-b6160930e009 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:35.212671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa463e3a-a87f-470a-8215-b6160930e009 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:35.245785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fdf0d-cae0-4265-804b-79c4c284061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:35.247446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fdf0d-cae0-4265-804b-79c4c284061b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:35.247956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fdf0d-cae0-4265-804b-79c4c284061b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:35.248686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fdf0d-cae0-4265-804b-79c4c284061b None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:35.250189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fdf0d-cae0-4265-804b-79c4c284061b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:35.250948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fdf0d-cae0-4265-804b-79c4c284061b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:35.252671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fdf0d-cae0-4265-804b-79c4c284061b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:35.278859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 327/662] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 8 18:33:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:35.279777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fdf0d-cae0-4265-804b-79c4c284061b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:35.341842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:35.346571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 336/663] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:35.352432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa463e3a-a87f-470a-8215-b6160930e009 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:35.393325 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fdf0d-cae0-4265-804b-79c4c284061b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:35.429271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50ecb-fa84-4f1a-b246-c5943f21cd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:35.429889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50ecb-fa84-4f1a-b246-c5943f21cd81 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:35.430174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50ecb-fa84-4f1a-b246-c5943f21cd81 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:35.430455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d50ecb-fa84-4f1a-b246-c5943f21cd81 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:35.431101 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d50ecb-fa84-4f1a-b246-c5943f21cd81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:35.431467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d50ecb-fa84-4f1a-b246-c5943f21cd81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:35.432875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d50ecb-fa84-4f1a-b246-c5943f21cd81 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:35.434515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:35.446060 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 328/664] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:35.446868 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50ecb-fa84-4f1a-b246-c5943f21cd81 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:35.467583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa463e3a-a87f-470a-8215-b6160930e009 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:35.505320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b276-ada1-4328-bf4b-9c8b9e933df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:35.506360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b276-ada1-4328-bf4b-9c8b9e933df1 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:35.506646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b276-ada1-4328-bf4b-9c8b9e933df1 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:35.507582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b276-ada1-4328-bf4b-9c8b9e933df1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:35.558619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d50ecb-fa84-4f1a-b246-c5943f21cd81 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:35.576455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:35.580179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9b276-ada1-4328-bf4b-9c8b9e933df1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:35.581664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9b276-ada1-4328-bf4b-9c8b9e933df1 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:35.600605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d12f0d-7f17-411f-bad0-5da94b34aec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:35.601150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d12f0d-7f17-411f-bad0-5da94b34aec5 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:35.601432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d12f0d-7f17-411f-bad0-5da94b34aec5 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:35.604612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d12f0d-7f17-411f-bad0-5da94b34aec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:35.613289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d12f0d-7f17-411f-bad0-5da94b34aec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:35.615088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d12f0d-7f17-411f-bad0-5da94b34aec5 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-31d12f0d-7f17-411f-bad0-5da94b34aec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:35.630614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:35.631973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:35.633245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:35.633245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:35.633245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:35.638896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 329/665] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:35.638896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d12f0d-7f17-411f-bad0-5da94b34aec5 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:35.688264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:35.701868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 337/666] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:35.703208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b276-ada1-4328-bf4b-9c8b9e933df1 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:35.752647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d12f0d-7f17-411f-bad0-5da94b34aec5 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:35.789505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecc4a5-8953-4383-bdf8-6b88dc300b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:35.790176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecc4a5-8953-4383-bdf8-6b88dc300b1c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:35.790908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecc4a5-8953-4383-bdf8-6b88dc300b1c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:35.791192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecc4a5-8953-4383-bdf8-6b88dc300b1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:35.803074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecc4a5-8953-4383-bdf8-6b88dc300b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-efecc4a5-8953-4383-bdf8-6b88dc300b1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:35.809538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:35.810506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:35.811341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 330/667] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:35.812046 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecc4a5-8953-4383-bdf8-6b88dc300b1c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:35.819389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:35.823532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b276-ada1-4328-bf4b-9c8b9e933df1 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:35.861071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fac0f-b25c-4514-8f2b-e8127dd62b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:35.861767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fac0f-b25c-4514-8f2b-e8127dd62b72 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:35.862049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fac0f-b25c-4514-8f2b-e8127dd62b72 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:35.862330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fac0f-b25c-4514-8f2b-e8127dd62b72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:35.870554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fac0f-b25c-4514-8f2b-e8127dd62b72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:35.872101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fac0f-b25c-4514-8f2b-e8127dd62b72 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-4f7fac0f-b25c-4514-8f2b-e8127dd62b72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:35.884667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:35.885615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:35.891481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:35.891481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:35.891481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:35.891481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:35.891481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:35.891481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:35.891481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:35.891481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:35.891481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 338/668] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:35.891481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fac0f-b25c-4514-8f2b-e8127dd62b72 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:35.918278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecc4a5-8953-4383-bdf8-6b88dc300b1c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:35.925993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:35.956550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b19d94-63c2-429d-bf17-c24f0b9d30d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:35.957358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b19d94-63c2-429d-bf17-c24f0b9d30d8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:35.957642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b19d94-63c2-429d-bf17-c24f0b9d30d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:35.957929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b19d94-63c2-429d-bf17-c24f0b9d30d8 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:35.958592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b19d94-63c2-429d-bf17-c24f0b9d30d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:35.958900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b19d94-63c2-429d-bf17-c24f0b9d30d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:35.960396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b19d94-63c2-429d-bf17-c24f0b9d30d8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:35.978680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 331/669] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:35.978680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b19d94-63c2-429d-bf17-c24f0b9d30d8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.011038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fac0f-b25c-4514-8f2b-e8127dd62b72 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.045909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbf5ac-d850-4ba1-8adf-35052a4b6829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:36.046543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbf5ac-d850-4ba1-8adf-35052a4b6829 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:36.046821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbf5ac-d850-4ba1-8adf-35052a4b6829 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:36.047092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbf5ac-d850-4ba1-8adf-35052a4b6829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:36.057433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbf5ac-d850-4ba1-8adf-35052a4b6829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:36.059023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbf5ac-d850-4ba1-8adf-35052a4b6829 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-3ffbf5ac-d850-4ba1-8adf-35052a4b6829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:36.074648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:36.075611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:36.076535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:36.076535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:36.076535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:36.076535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 339/670] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:36.076535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbf5ac-d850-4ba1-8adf-35052a4b6829 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.089429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:36.101697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b19d94-63c2-429d-bf17-c24f0b9d30d8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.137314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699765b6-563d-44a7-929d-652b5cf922ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:36.137850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699765b6-563d-44a7-929d-652b5cf922ac None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:36.138142 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699765b6-563d-44a7-929d-652b5cf922ac None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:36.138415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699765b6-563d-44a7-929d-652b5cf922ac None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:36.139083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699765b6-563d-44a7-929d-652b5cf922ac None admin] RBAC: Adding query filter params (name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:36.139398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699765b6-563d-44a7-929d-652b5cf922ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:36.140759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699765b6-563d-44a7-929d-652b5cf922ac None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:36.153249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 332/671] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:36.153249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699765b6-563d-44a7-929d-652b5cf922ac None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.180669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:36.186955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbf5ac-d850-4ba1-8adf-35052a4b6829 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.221305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5041602-ca43-4c48-ae28-0b48dc9a9f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:36.221912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5041602-ca43-4c48-ae28-0b48dc9a9f5a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:36.222189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5041602-ca43-4c48-ae28-0b48dc9a9f5a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:36.222467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5041602-ca43-4c48-ae28-0b48dc9a9f5a None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:36.223141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5041602-ca43-4c48-ae28-0b48dc9a9f5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:36.223447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5041602-ca43-4c48-ae28-0b48dc9a9f5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:36.226592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5041602-ca43-4c48-ae28-0b48dc9a9f5a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:36.239938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 340/672] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:36.240591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5041602-ca43-4c48-ae28-0b48dc9a9f5a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.267114 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699765b6-563d-44a7-929d-652b5cf922ac None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.305559 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8188cf3-c56e-4225-817b-88ea80fb8ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:36.306185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8188cf3-c56e-4225-817b-88ea80fb8ca2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:36.306461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8188cf3-c56e-4225-817b-88ea80fb8ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:36.306731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8188cf3-c56e-4225-817b-88ea80fb8ca2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:36.313691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8188cf3-c56e-4225-817b-88ea80fb8ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:36.315134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8188cf3-c56e-4225-817b-88ea80fb8ca2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-c8188cf3-c56e-4225-817b-88ea80fb8ca2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:36.327520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:36.328620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:36.329564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:36.329564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:36.329564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:36.329564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:36.329564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:36.329564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:36.329564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:36.329564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:36.329564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 333/673] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:36.330063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8188cf3-c56e-4225-817b-88ea80fb8ca2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.338898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:36.373428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5041602-ca43-4c48-ae28-0b48dc9a9f5a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.410361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c4755-c276-4ddd-8e72-4359e661725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:36.410888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c4755-c276-4ddd-8e72-4359e661725b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:36.411169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c4755-c276-4ddd-8e72-4359e661725b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:36.411433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c4755-c276-4ddd-8e72-4359e661725b None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:36.412172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c4755-c276-4ddd-8e72-4359e661725b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:36.412477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c4755-c276-4ddd-8e72-4359e661725b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:36.413891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c4755-c276-4ddd-8e72-4359e661725b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:36.425361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 341/674] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:36.426173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c4755-c276-4ddd-8e72-4359e661725b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.436312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:36.450066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8188cf3-c56e-4225-817b-88ea80fb8ca2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.487839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67328a-a522-48e3-977a-e115acdb05c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:36.488687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67328a-a522-48e3-977a-e115acdb05c8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:36.489107 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67328a-a522-48e3-977a-e115acdb05c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:36.489397 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67328a-a522-48e3-977a-e115acdb05c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:36.500321 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67328a-a522-48e3-977a-e115acdb05c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:36.501913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67328a-a522-48e3-977a-e115acdb05c8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-6b67328a-a522-48e3-977a-e115acdb05c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:36.517381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:36.518349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:36.519254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:36.519254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:36.519254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:36.519254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 334/675] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:36.519497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67328a-a522-48e3-977a-e115acdb05c8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.535951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c4755-c276-4ddd-8e72-4359e661725b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.564852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:36.573809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d00fba-708f-412a-9352-a53e578a1a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:36.578483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d00fba-708f-412a-9352-a53e578a1a4a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:36.578483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d00fba-708f-412a-9352-a53e578a1a4a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:36.578483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d00fba-708f-412a-9352-a53e578a1a4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:36.578483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d00fba-708f-412a-9352-a53e578a1a4a None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=605b5d4c0fc9409a94dad22c77c67332) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:36.578483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d00fba-708f-412a-9352-a53e578a1a4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:36.578483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d00fba-708f-412a-9352-a53e578a1a4a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:36.593673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 342/676] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:36 2022] GET /identity/v3/role_assignments?user.id=605b5d4c0fc9409a94dad22c77c67332&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:36.594591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d00fba-708f-412a-9352-a53e578a1a4a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.650620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67328a-a522-48e3-977a-e115acdb05c8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.692904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2470b0-4952-4a82-ad91-55d70d61c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:36.693483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2470b0-4952-4a82-ad91-55d70d61c97b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:36.693766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2470b0-4952-4a82-ad91-55d70d61c97b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:36.694103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2470b0-4952-4a82-ad91-55d70d61c97b None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:36.695532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2470b0-4952-4a82-ad91-55d70d61c97b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:36.695851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2470b0-4952-4a82-ad91-55d70d61c97b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:36.697442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2470b0-4952-4a82-ad91-55d70d61c97b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:36.708231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:36.717187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 335/677] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:36.718459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2470b0-4952-4a82-ad91-55d70d61c97b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.758014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d00fba-708f-412a-9352-a53e578a1a4a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.802599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7c212-393c-45cc-905a-836cb7df059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:36.803240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7c212-393c-45cc-905a-836cb7df059c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:36.803784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7c212-393c-45cc-905a-836cb7df059c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:36.804066 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7c212-393c-45cc-905a-836cb7df059c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:36.812064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7c212-393c-45cc-905a-836cb7df059c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:36.813552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7c212-393c-45cc-905a-836cb7df059c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-b5e7c212-393c-45cc-905a-836cb7df059c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:36.828764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:36.829753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:36.830645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:36.830645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:36.830645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:36.830645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:36.830645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:36.830645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:36.830645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:36.830645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:36.831635 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 343/678] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:36.832127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7c212-393c-45cc-905a-836cb7df059c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.847948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2470b0-4952-4a82-ad91-55d70d61c97b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.891418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dea494-372e-4c12-a665-02f67acb990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:36.892464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea494-372e-4c12-a665-02f67acb990d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:36.892845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea494-372e-4c12-a665-02f67acb990d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:36.893154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea494-372e-4c12-a665-02f67acb990d None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:36.893808 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea494-372e-4c12-a665-02f67acb990d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:36.894115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea494-372e-4c12-a665-02f67acb990d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:36.896141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea494-372e-4c12-a665-02f67acb990d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:36.898784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:36.913809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 336/679] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:36.915176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dea494-372e-4c12-a665-02f67acb990d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:36.961154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7c212-393c-45cc-905a-836cb7df059c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:36.991006 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:37.005188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fa95e-3382-4b8e-bd6a-987b9c45b169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:37.006261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fa95e-3382-4b8e-bd6a-987b9c45b169 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:37.007218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fa95e-3382-4b8e-bd6a-987b9c45b169 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:37.007640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fa95e-3382-4b8e-bd6a-987b9c45b169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:37.008515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1fa95e-3382-4b8e-bd6a-987b9c45b169 None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=c8d10585480f41608bc6935e282f7e27) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:37.009065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1fa95e-3382-4b8e-bd6a-987b9c45b169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:37.010752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1fa95e-3382-4b8e-bd6a-987b9c45b169 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:37.017593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 344/680] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:36 2022] GET /identity/v3/role_assignments?user.id=c8d10585480f41608bc6935e282f7e27&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:37.019284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fa95e-3382-4b8e-bd6a-987b9c45b169 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:37.084039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dea494-372e-4c12-a665-02f67acb990d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:37.143370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fa95e-3382-4b8e-bd6a-987b9c45b169 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:37.164729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:37.171179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e4d46-9d83-44c8-9d26-909e4546b5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:37.171823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e4d46-9d83-44c8-9d26-909e4546b5fb None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:37.172158 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e4d46-9d83-44c8-9d26-909e4546b5fb None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:37.172435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e4d46-9d83-44c8-9d26-909e4546b5fb None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:37.173256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e4d46-9d83-44c8-9d26-909e4546b5fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:37.176065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e4d46-9d83-44c8-9d26-909e4546b5fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:37.176065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e4d46-9d83-44c8-9d26-909e4546b5fb None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:37.188790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd65ff1-3cc9-4b12-a3d2-8c2a7288357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:37.189874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd65ff1-3cc9-4b12-a3d2-8c2a7288357e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:37.190318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd65ff1-3cc9-4b12-a3d2-8c2a7288357e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:37.190888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd65ff1-3cc9-4b12-a3d2-8c2a7288357e None admin] PATH_INFO: `/v3/users/swift` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:37.201266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd65ff1-3cc9-4b12-a3d2-8c2a7288357e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:37.202717 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 337/681] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:37.203192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd65ff1-3cc9-4b12-a3d2-8c2a7288357e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:37.210859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e4d46-9d83-44c8-9d26-909e4546b5fb None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-5dd65ff1-3cc9-4b12-a3d2-8c2a7288357e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:37.225294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:37.226711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:37.227695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:37.227695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:37.227695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:37.227695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:37.227695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:37.227695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:37.227695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:37.227695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:37.227695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:37.227695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:37.230919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 345/682] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:37.241851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd65ff1-3cc9-4b12-a3d2-8c2a7288357e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:37.364362 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e4d46-9d83-44c8-9d26-909e4546b5fb None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:37.389627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:37.399154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd65ff1-3cc9-4b12-a3d2-8c2a7288357e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:37.415724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab78c37-79a1-4137-b8f8-dba5d437265a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:37.416873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab78c37-79a1-4137-b8f8-dba5d437265a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:37.418556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab78c37-79a1-4137-b8f8-dba5d437265a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:37.418986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab78c37-79a1-4137-b8f8-dba5d437265a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:37.419866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab78c37-79a1-4137-b8f8-dba5d437265a None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=bcf30ef695154d3a97d7e589aca21233) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:37.420322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab78c37-79a1-4137-b8f8-dba5d437265a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:37.422796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab78c37-79a1-4137-b8f8-dba5d437265a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:37.433419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 338/683] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:37 2022] GET /identity/v3/role_assignments?user.id=bcf30ef695154d3a97d7e589aca21233&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:37.435177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71eddc-cddd-4d32-a0fa-577cb30cb962 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:37.435630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71eddc-cddd-4d32-a0fa-577cb30cb962 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:37.436049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71eddc-cddd-4d32-a0fa-577cb30cb962 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:37.441071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e762be8-a891-4a5c-b950-5a4debc813d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:37.441071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e762be8-a891-4a5c-b950-5a4debc813d5 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:37.441071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e762be8-a891-4a5c-b950-5a4debc813d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:37.441071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e762be8-a891-4a5c-b950-5a4debc813d5 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:37.441517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e762be8-a891-4a5c-b950-5a4debc813d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:37.441517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e762be8-a891-4a5c-b950-5a4debc813d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:37.441517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e762be8-a891-4a5c-b950-5a4debc813d5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:37.443941 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 339/684] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:37.450713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c454912-48ad-4fed-9a56-eef6347aff44 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:37.451048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c454912-48ad-4fed-9a56-eef6347aff44 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:37.454154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c454912-48ad-4fed-9a56-eef6347aff44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:37.481654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 346/685] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 18:33:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:37.487644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e762be8-a891-4a5c-b950-5a4debc813d5 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:37.492924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-8c454912-48ad-4fed-9a56-eef6347aff44 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:37.497492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8c454912-48ad-4fed-9a56-eef6347aff44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:37.559095 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:37.645196 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 340/686] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:33:37.683381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e762be8-a891-4a5c-b950-5a4debc813d5 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:37.721652 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688685f-193a-4dfd-9ee6-3614e7316d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:37.722350 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688685f-193a-4dfd-9ee6-3614e7316d9e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:37.722632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688685f-193a-4dfd-9ee6-3614e7316d9e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:37.722906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688685f-193a-4dfd-9ee6-3614e7316d9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:37.805820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688685f-193a-4dfd-9ee6-3614e7316d9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:37.807342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688685f-193a-4dfd-9ee6-3614e7316d9e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:37.842782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:37.957322 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 347/687] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:37.963617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c454912-48ad-4fed-9a56-eef6347aff44 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:37.980630 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688685f-193a-4dfd-9ee6-3614e7316d9e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:38.089305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688685f-193a-4dfd-9ee6-3614e7316d9e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:38.125363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2de539-c2ff-4e18-bcdb-0d81b972513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:38.126263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2de539-c2ff-4e18-bcdb-0d81b972513d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:38.126723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2de539-c2ff-4e18-bcdb-0d81b972513d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:38.127126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2de539-c2ff-4e18-bcdb-0d81b972513d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:38.133423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:38.141726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2de539-c2ff-4e18-bcdb-0d81b972513d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-cf2de539-c2ff-4e18-bcdb-0d81b972513d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:38.154300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:38.155395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:38.156471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 348/688] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:38.160832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2de539-c2ff-4e18-bcdb-0d81b972513d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:38.183946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c454912-48ad-4fed-9a56-eef6347aff44 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:38.213418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:38.232600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ee737-fe80-40dc-867e-b92eff123460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:38.233272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee737-fe80-40dc-867e-b92eff123460 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:38.233553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee737-fe80-40dc-867e-b92eff123460 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:38.233830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee737-fe80-40dc-867e-b92eff123460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:38.260636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ee737-fe80-40dc-867e-b92eff123460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:38.262148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ee737-fe80-40dc-867e-b92eff123460 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-5b2ee737-fe80-40dc-867e-b92eff123460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:38.294224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:38.295263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:38.296177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:38.296177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:38.296177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:38.296177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 341/689] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:38.301368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ee737-fe80-40dc-867e-b92eff123460 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:38.318667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2de539-c2ff-4e18-bcdb-0d81b972513d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:38.372446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0f1c4-20d0-4cdf-9b94-7a9511137306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:38.372996 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0f1c4-20d0-4cdf-9b94-7a9511137306 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:38.373305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0f1c4-20d0-4cdf-9b94-7a9511137306 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:38.373577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0f1c4-20d0-4cdf-9b94-7a9511137306 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:38.374248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0f1c4-20d0-4cdf-9b94-7a9511137306 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:38.374663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0f1c4-20d0-4cdf-9b94-7a9511137306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:38.376144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0f1c4-20d0-4cdf-9b94-7a9511137306 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:38.385953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 349/690] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:38.394232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0f1c4-20d0-4cdf-9b94-7a9511137306 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:38.447897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ee737-fe80-40dc-867e-b92eff123460 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:38.477616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:38.517790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53356d0e-2003-4afc-bcc5-52a2213f7481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:38.518405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53356d0e-2003-4afc-bcc5-52a2213f7481 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:38.518684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53356d0e-2003-4afc-bcc5-52a2213f7481 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:38.518958 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53356d0e-2003-4afc-bcc5-52a2213f7481 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:38.519603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53356d0e-2003-4afc-bcc5-52a2213f7481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:38.519905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53356d0e-2003-4afc-bcc5-52a2213f7481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:38.521626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0f1c4-20d0-4cdf-9b94-7a9511137306 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:38.527405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53356d0e-2003-4afc-bcc5-52a2213f7481 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:38.555581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 342/691] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:38.560643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53356d0e-2003-4afc-bcc5-52a2213f7481 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:38.568377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f32c0-fea9-443c-bbbc-685a10585ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:38.569690 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f32c0-fea9-443c-bbbc-685a10585ee3 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:38.569986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f32c0-fea9-443c-bbbc-685a10585ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:38.570265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f32c0-fea9-443c-bbbc-685a10585ee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:38.592164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f32c0-fea9-443c-bbbc-685a10585ee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:38.593959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f32c0-fea9-443c-bbbc-685a10585ee3 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:38.600600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-c17f32c0-fea9-443c-bbbc-685a10585ee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:38.614446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:38.620331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:38.621258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:38.621258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:38.621258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:38.621258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 350/692] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:38.625307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f32c0-fea9-443c-bbbc-685a10585ee3 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:38.743296 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f32c0-fea9-443c-bbbc-685a10585ee3 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:38.754492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:38.756603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53356d0e-2003-4afc-bcc5-52a2213f7481 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:38.779632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e2969-dbba-49dc-a8a6-bd8f8e9d1186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:38.780330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e2969-dbba-49dc-a8a6-bd8f8e9d1186 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:38.780670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e2969-dbba-49dc-a8a6-bd8f8e9d1186 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:38.785684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e2969-dbba-49dc-a8a6-bd8f8e9d1186 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:38.785684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e2969-dbba-49dc-a8a6-bd8f8e9d1186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:38.785684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e2969-dbba-49dc-a8a6-bd8f8e9d1186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:38.785684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e2969-dbba-49dc-a8a6-bd8f8e9d1186 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:38.796877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 351/693] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:38.805306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e2969-dbba-49dc-a8a6-bd8f8e9d1186 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:38.829104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99573f1f-d3b6-4cc6-8629-0db87f0bb8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:38.829758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99573f1f-d3b6-4cc6-8629-0db87f0bb8fe None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:38.830041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99573f1f-d3b6-4cc6-8629-0db87f0bb8fe None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:38.830312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99573f1f-d3b6-4cc6-8629-0db87f0bb8fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:38.839747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99573f1f-d3b6-4cc6-8629-0db87f0bb8fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:38.841243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99573f1f-d3b6-4cc6-8629-0db87f0bb8fe None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-99573f1f-d3b6-4cc6-8629-0db87f0bb8fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:38.854997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:38.855919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:38.856798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:38.856798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:38.856798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:38.856798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:38.856798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:38.856798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:38.856798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:38.856798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:38.856798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 343/694] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:38.862947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99573f1f-d3b6-4cc6-8629-0db87f0bb8fe None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:39.002887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e2969-dbba-49dc-a8a6-bd8f8e9d1186 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:39.033615 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:39.065145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cbda78-a8b1-4ec1-8730-55ef85c983de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:39.066130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbda78-a8b1-4ec1-8730-55ef85c983de None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.066626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbda78-a8b1-4ec1-8730-55ef85c983de None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.067093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbda78-a8b1-4ec1-8730-55ef85c983de None admin] PATH_INFO: `/v3/users/nova` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:39.067199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99573f1f-d3b6-4cc6-8629-0db87f0bb8fe None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:39.075772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cbda78-a8b1-4ec1-8730-55ef85c983de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:39.077572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cbda78-a8b1-4ec1-8730-55ef85c983de None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-d2cbda78-a8b1-4ec1-8730-55ef85c983de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:39.089745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:39.091138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:39.092071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:39.092071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:39.092071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:39.092071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:39.092071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:39.092071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:39.092071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:39.092071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:39.092071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:39.092071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:39.094934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 352/695] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 18:33:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:39.101025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cbda78-a8b1-4ec1-8730-55ef85c983de None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:39.144073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213d707-3539-48d2-ad5a-5c03d2978a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:39.144773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213d707-3539-48d2-ad5a-5c03d2978a88 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.145083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213d707-3539-48d2-ad5a-5c03d2978a88 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.145372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213d707-3539-48d2-ad5a-5c03d2978a88 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:39.146186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213d707-3539-48d2-ad5a-5c03d2978a88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:39.146385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213d707-3539-48d2-ad5a-5c03d2978a88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:39.148087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213d707-3539-48d2-ad5a-5c03d2978a88 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:39.151783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:39.164758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 344/696] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:39.166977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f94e0-10e6-457b-a8c2-7cabdadb10cd None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.167593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f94e0-10e6-457b-a8c2-7cabdadb10cd None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.168023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f94e0-10e6-457b-a8c2-7cabdadb10cd None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:39.174407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 345/697] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:39.174407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f94e0-10e6-457b-a8c2-7cabdadb10cd None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:39.222293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cbda78-a8b1-4ec1-8730-55ef85c983de None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:39.266514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2aaf2-1351-4b5e-87fa-f69aa304aa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:39.267621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2aaf2-1351-4b5e-87fa-f69aa304aa50 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.268263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2aaf2-1351-4b5e-87fa-f69aa304aa50 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.268606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2aaf2-1351-4b5e-87fa-f69aa304aa50 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:39.269446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2aaf2-1351-4b5e-87fa-f69aa304aa50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:39.269827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2aaf2-1351-4b5e-87fa-f69aa304aa50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:39.271421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2aaf2-1351-4b5e-87fa-f69aa304aa50 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:39.298224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 353/698] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 18:33:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:39.298224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba774bd5-32ba-48d9-8f0a-c6847905465a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.298224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba774bd5-32ba-48d9-8f0a-c6847905465a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.298224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba774bd5-32ba-48d9-8f0a-c6847905465a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:39.310862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f94e0-10e6-457b-a8c2-7cabdadb10cd None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:39.318151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-ba774bd5-32ba-48d9-8f0a-c6847905465a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:39.320095 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ba774bd5-32ba-48d9-8f0a-c6847905465a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:39.356448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b894f2-997c-4cd4-aaf8-c9292389d9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:39.357709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b894f2-997c-4cd4-aaf8-c9292389d9fa None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.358206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b894f2-997c-4cd4-aaf8-c9292389d9fa None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.358928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b894f2-997c-4cd4-aaf8-c9292389d9fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:39.360270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b894f2-997c-4cd4-aaf8-c9292389d9fa None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=88a9b2cee21f4831b221d113ddbbd29d) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:39.361025 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b894f2-997c-4cd4-aaf8-c9292389d9fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:39.363003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b894f2-997c-4cd4-aaf8-c9292389d9fa None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:39.373792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 346/699] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:39 2022] GET /identity/v3/role_assignments?user.id=88a9b2cee21f4831b221d113ddbbd29d&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:39.373792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b894f2-997c-4cd4-aaf8-c9292389d9fa None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:39.390013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:39.466856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 354/700] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:39.503866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b894f2-997c-4cd4-aaf8-c9292389d9fa None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:39.549856 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e767d-fcf5-422d-909b-2ce8a728720a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:39.550493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e767d-fcf5-422d-909b-2ce8a728720a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.550774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e767d-fcf5-422d-909b-2ce8a728720a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.551048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e767d-fcf5-422d-909b-2ce8a728720a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:39.643687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e767d-fcf5-422d-909b-2ce8a728720a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:39.645239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e767d-fcf5-422d-909b-2ce8a728720a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:39.657804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:39.754902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 347/701] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:39.759348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba774bd5-32ba-48d9-8f0a-c6847905465a None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:39.760532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e767d-fcf5-422d-909b-2ce8a728720a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:39.784078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:39.885980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e767d-fcf5-422d-909b-2ce8a728720a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:39.892480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba774bd5-32ba-48d9-8f0a-c6847905465a None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:39.903484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:39.942253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5020e6-9de6-40a0-b3d1-9d2562a77d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:39.943134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5020e6-9de6-40a0-b3d1-9d2562a77d02 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.943555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5020e6-9de6-40a0-b3d1-9d2562a77d02 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.944937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5020e6-9de6-40a0-b3d1-9d2562a77d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:39.946507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ea74a-55c6-4ba5-9ceb-f7d7e22f6b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:39.947159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ea74a-55c6-4ba5-9ceb-f7d7e22f6b54 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.947437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ea74a-55c6-4ba5-9ceb-f7d7e22f6b54 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.947958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ea74a-55c6-4ba5-9ceb-f7d7e22f6b54 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:39.948556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ea74a-55c6-4ba5-9ceb-f7d7e22f6b54 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:39.950236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ea74a-55c6-4ba5-9ceb-f7d7e22f6b54 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:39.957859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5020e6-9de6-40a0-b3d1-9d2562a77d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:39.959070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5020e6-9de6-40a0-b3d1-9d2562a77d02 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-c72ea74a-55c6-4ba5-9ceb-f7d7e22f6b54 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:39.961566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:39.962643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:39.963537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 355/702] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 8 18:33:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:39.964933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc633a1-d4d1-4bc1-bb76-f0ffbabee820 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.965191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc633a1-d4d1-4bc1-bb76-f0ffbabee820 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.965434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc633a1-d4d1-4bc1-bb76-f0ffbabee820 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:39.969080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 356/703] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:39.969080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc633a1-d4d1-4bc1-bb76-f0ffbabee820 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-8e5020e6-9de6-40a0-b3d1-9d2562a77d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:39.981790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:39.982766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:39.983657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:39.983657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:39.983657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:39.983657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 348/704] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:39.984900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775a9a7-f4fd-47fd-9ae8-1490768cdb72 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:39.984900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775a9a7-f4fd-47fd-9ae8-1490768cdb72 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:39.985510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775a9a7-f4fd-47fd-9ae8-1490768cdb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:40.004510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-9775a9a7-f4fd-47fd-9ae8-1490768cdb72 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:40.006276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9775a9a7-f4fd-47fd-9ae8-1490768cdb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:40.073728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:40.096105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 349/705] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:40.096105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775a9a7-f4fd-47fd-9ae8-1490768cdb72 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:40.113761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc633a1-d4d1-4bc1-bb76-f0ffbabee820 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:40.170352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00ff40-1601-41b3-ae88-e09c59669891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:40.171421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00ff40-1601-41b3-ae88-e09c59669891 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:40.171955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00ff40-1601-41b3-ae88-e09c59669891 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:40.172480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00ff40-1601-41b3-ae88-e09c59669891 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:40.173387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00ff40-1601-41b3-ae88-e09c59669891 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:40.173944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00ff40-1601-41b3-ae88-e09c59669891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:40.176791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00ff40-1601-41b3-ae88-e09c59669891 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:40.183379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 357/706] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 18:33:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:40.185781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b884-413b-42d3-8978-9033a5ca9460 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:40.186254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b884-413b-42d3-8978-9033a5ca9460 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:40.186963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b884-413b-42d3-8978-9033a5ca9460 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:40.188707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 358/707] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:40.189713 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b884-413b-42d3-8978-9033a5ca9460 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:40.202275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:40.212473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775a9a7-f4fd-47fd-9ae8-1490768cdb72 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:40.254973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7010e-750e-4ce2-a576-7229c1c37470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:40.256022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7010e-750e-4ce2-a576-7229c1c37470 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:40.256485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7010e-750e-4ce2-a576-7229c1c37470 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:40.257560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7010e-750e-4ce2-a576-7229c1c37470 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:40.259029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7010e-750e-4ce2-a576-7229c1c37470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:40.259523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7010e-750e-4ce2-a576-7229c1c37470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:40.261739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7010e-750e-4ce2-a576-7229c1c37470 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:40.283318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 350/708] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:40.283754 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30862d-5db2-4386-9319-c6f2847eb6b5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:40.284056 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30862d-5db2-4386-9319-c6f2847eb6b5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:40.284340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30862d-5db2-4386-9319-c6f2847eb6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:40.303182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2b30862d-5db2-4386-9319-c6f2847eb6b5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:40.305261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2b30862d-5db2-4386-9319-c6f2847eb6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:40.312502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b884-413b-42d3-8978-9033a5ca9460 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:40.355029 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a278fd-26f6-406c-94ae-976fdcd38b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:40.355754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a278fd-26f6-406c-94ae-976fdcd38b75 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:40.356118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a278fd-26f6-406c-94ae-976fdcd38b75 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:40.356460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a278fd-26f6-406c-94ae-976fdcd38b75 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:40.357192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a278fd-26f6-406c-94ae-976fdcd38b75 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:40.357566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a278fd-26f6-406c-94ae-976fdcd38b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:40.359257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a278fd-26f6-406c-94ae-976fdcd38b75 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:40.366925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 359/709] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 18:33:40 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:40.368078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a278fd-26f6-406c-94ae-976fdcd38b75 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:40.389254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 351/710] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:40.390432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:40.441313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30862d-5db2-4386-9319-c6f2847eb6b5 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:40.489059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a278fd-26f6-406c-94ae-976fdcd38b75 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:40.531482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32581498-c0a3-4fe3-93a7-a96be8167c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:40.532156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32581498-c0a3-4fe3-93a7-a96be8167c53 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:40.532439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32581498-c0a3-4fe3-93a7-a96be8167c53 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:40.532717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32581498-c0a3-4fe3-93a7-a96be8167c53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:40.541599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32581498-c0a3-4fe3-93a7-a96be8167c53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:40.543102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32581498-c0a3-4fe3-93a7-a96be8167c53 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-32581498-c0a3-4fe3-93a7-a96be8167c53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:40.558915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:40.560149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:40.561469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:40.561469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:40.561469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:40.561469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:40.561469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:40.561469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:40.561469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:40.561469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:40.561469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 360/711] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:40.567247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32581498-c0a3-4fe3-93a7-a96be8167c53 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:40.626844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:40.652637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30862d-5db2-4386-9319-c6f2847eb6b5 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:40.702611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32447b7c-adbc-469b-a812-3ab53301a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:40.703517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32447b7c-adbc-469b-a812-3ab53301a8cf None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:40.703982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32447b7c-adbc-469b-a812-3ab53301a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:40.704412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32447b7c-adbc-469b-a812-3ab53301a8cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:40.722573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32447b7c-adbc-469b-a812-3ab53301a8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:40.724397 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32447b7c-adbc-469b-a812-3ab53301a8cf None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-32447b7c-adbc-469b-a812-3ab53301a8cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:40.759068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:40.760217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:40.761247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:40.761247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:40.761247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:40.763720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 352/712] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:40.770360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32447b7c-adbc-469b-a812-3ab53301a8cf None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:40.806216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:40.840094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32581498-c0a3-4fe3-93a7-a96be8167c53 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:40.876761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1f992-1d79-47bc-a6e0-6940ee61ea80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:40.877768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1f992-1d79-47bc-a6e0-6940ee61ea80 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:40.878256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1f992-1d79-47bc-a6e0-6940ee61ea80 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:40.878736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1f992-1d79-47bc-a6e0-6940ee61ea80 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:40.880123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1f992-1d79-47bc-a6e0-6940ee61ea80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:40.880778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1f992-1d79-47bc-a6e0-6940ee61ea80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:40.883467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1f992-1d79-47bc-a6e0-6940ee61ea80 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:40.886906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32447b7c-adbc-469b-a812-3ab53301a8cf None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:40.898890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 361/713] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:40.900138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1f992-1d79-47bc-a6e0-6940ee61ea80 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:40.933334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c450f5-6688-4fbc-aa00-59f69042f838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:40.934484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c450f5-6688-4fbc-aa00-59f69042f838 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:40.934958 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c450f5-6688-4fbc-aa00-59f69042f838 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:40.937191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c450f5-6688-4fbc-aa00-59f69042f838 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:40.937191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c450f5-6688-4fbc-aa00-59f69042f838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:40.937191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c450f5-6688-4fbc-aa00-59f69042f838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:40.938212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c450f5-6688-4fbc-aa00-59f69042f838 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:40.940790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:40.953149 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 353/714] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:40.954324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c450f5-6688-4fbc-aa00-59f69042f838 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:41.017831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1f992-1d79-47bc-a6e0-6940ee61ea80 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:41.056303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:41.064059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487889fd-8cdc-47d1-9201-8bb66900dc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:41.064901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487889fd-8cdc-47d1-9201-8bb66900dc30 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:41.065374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487889fd-8cdc-47d1-9201-8bb66900dc30 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:41.065706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487889fd-8cdc-47d1-9201-8bb66900dc30 None admin] PATH_INFO: `/v3/services/network` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:41.066365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487889fd-8cdc-47d1-9201-8bb66900dc30 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:41.068524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487889fd-8cdc-47d1-9201-8bb66900dc30 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-487889fd-8cdc-47d1-9201-8bb66900dc30 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:41.081414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:41.082700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 18:33:41.083859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:41.087057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 362/715] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:40 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:41.088700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487889fd-8cdc-47d1-9201-8bb66900dc30 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:41.095089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c450f5-6688-4fbc-aa00-59f69042f838 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:41.143820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1a045-01c3-41e2-8274-3e4e7e0bb353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:41.145668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1a045-01c3-41e2-8274-3e4e7e0bb353 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:41.146288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1a045-01c3-41e2-8274-3e4e7e0bb353 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:41.146790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1a045-01c3-41e2-8274-3e4e7e0bb353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:41.149292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1a045-01c3-41e2-8274-3e4e7e0bb353 None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=1b29c5745a8e42ff8232e5ca22b4a4ad) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:41.149966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1a045-01c3-41e2-8274-3e4e7e0bb353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:41.152120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1a045-01c3-41e2-8274-3e4e7e0bb353 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:41.161082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 354/716] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:40 2022] GET /identity/v3/role_assignments?user.id=1b29c5745a8e42ff8232e5ca22b4a4ad&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=5c777de40b8449329d714d8c2229115e => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:41.162496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1a045-01c3-41e2-8274-3e4e7e0bb353 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:41.219249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487889fd-8cdc-47d1-9201-8bb66900dc30 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:41.233096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:41.273726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac56f3-c964-40c2-94d6-f2353b21efa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:41.275373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac56f3-c964-40c2-94d6-f2353b21efa5 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:41.275671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac56f3-c964-40c2-94d6-f2353b21efa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:41.275952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac56f3-c964-40c2-94d6-f2353b21efa5 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:41.278062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac56f3-c964-40c2-94d6-f2353b21efa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:41.279541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac56f3-c964-40c2-94d6-f2353b21efa5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:41.297402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1a045-01c3-41e2-8274-3e4e7e0bb353 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:41.341495 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8d4e2-f9f4-4059-8bc5-43350585a807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:41.342229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8d4e2-f9f4-4059-8bc5-43350585a807 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:41.342604 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8d4e2-f9f4-4059-8bc5-43350585a807 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:41.342885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8d4e2-f9f4-4059-8bc5-43350585a807 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:41.343905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8d4e2-f9f4-4059-8bc5-43350585a807 None admin] RBAC: Adding query filter params (name=network) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:41.344314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8d4e2-f9f4-4059-8bc5-43350585a807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:41.346049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8d4e2-f9f4-4059-8bc5-43350585a807 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:41.353173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 355/717] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:33:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:41.358470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8d4e2-f9f4-4059-8bc5-43350585a807 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:41.380968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 363/718] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:33:41 2022] POST /identity/v3/users => generated 254 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:33:41.519166 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8d4e2-f9f4-4059-8bc5-43350585a807 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:41.545653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:41.560365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674cb4b-cdec-4c0e-9860-36f46da927c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:41.560978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674cb4b-cdec-4c0e-9860-36f46da927c0 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:41.561309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674cb4b-cdec-4c0e-9860-36f46da927c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:41.561582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674cb4b-cdec-4c0e-9860-36f46da927c0 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:41.562284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674cb4b-cdec-4c0e-9860-36f46da927c0 None admin] RBAC: Adding query filter params (type=network) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:41.562644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674cb4b-cdec-4c0e-9860-36f46da927c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:41.564020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674cb4b-cdec-4c0e-9860-36f46da927c0 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:41.571111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 356/719] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:33:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:42.107986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959e76a-5401-4d0b-91d5-20fd60715815 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:42.108271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959e76a-5401-4d0b-91d5-20fd60715815 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:42.108553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959e76a-5401-4d0b-91d5-20fd60715815 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:42.110151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 364/720] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:42.115699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfdd6c-e04d-45e5-b6c6-45faeaa943cc None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:42.116044 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfdd6c-e04d-45e5-b6c6-45faeaa943cc None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:42.116360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfdd6c-e04d-45e5-b6c6-45faeaa943cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:42.135502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c8dfdd6c-e04d-45e5-b6c6-45faeaa943cc None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:42.136703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c8dfdd6c-e04d-45e5-b6c6-45faeaa943cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:42.207491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 357/721] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:42.484121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959e76a-5401-4d0b-91d5-20fd60715815 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:42.539281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:42.592231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959e76a-5401-4d0b-91d5-20fd60715815 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:42.642875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd4f7d-a3bf-4b31-8730-99e066da86c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:42.643516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd4f7d-a3bf-4b31-8730-99e066da86c1 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:42.643803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd4f7d-a3bf-4b31-8730-99e066da86c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:42.644081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd4f7d-a3bf-4b31-8730-99e066da86c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:42.656187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd4f7d-a3bf-4b31-8730-99e066da86c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-04cd4f7d-a3bf-4b31-8730-99e066da86c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:42.661136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:42.662071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:42.664159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 365/722] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:42.668955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfdd6c-e04d-45e5-b6c6-45faeaa943cc None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:42.808884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfdd6c-e04d-45e5-b6c6-45faeaa943cc None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:42.828340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:42.854042 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b8186-7b99-4390-a5c9-b61ddc9c93f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:42.854657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b8186-7b99-4390-a5c9-b61ddc9c93f6 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:42.854932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b8186-7b99-4390-a5c9-b61ddc9c93f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:42.855203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b8186-7b99-4390-a5c9-b61ddc9c93f6 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:42.855875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b8186-7b99-4390-a5c9-b61ddc9c93f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:42.856177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b8186-7b99-4390-a5c9-b61ddc9c93f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:42.857655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b8186-7b99-4390-a5c9-b61ddc9c93f6 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:42.872977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 358/723] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:42.878296 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd4f7d-a3bf-4b31-8730-99e066da86c1 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:43.011610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd4f7d-a3bf-4b31-8730-99e066da86c1 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:43.022382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:43.046064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35153664-974f-4359-915e-4dd362687c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:43.046749 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35153664-974f-4359-915e-4dd362687c2c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:43.047033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35153664-974f-4359-915e-4dd362687c2c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:43.047305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35153664-974f-4359-915e-4dd362687c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:43.058332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35153664-974f-4359-915e-4dd362687c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:43.059883 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35153664-974f-4359-915e-4dd362687c2c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-35153664-974f-4359-915e-4dd362687c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:43.075262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:43.076320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:43.077862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:43.077862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:43.077862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:43.077862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 366/724] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:43.082270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b8186-7b99-4390-a5c9-b61ddc9c93f6 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:43.207057 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b8186-7b99-4390-a5c9-b61ddc9c93f6 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:43.250617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42916a4c-509a-4f8a-baed-249168b5d64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:43.252030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42916a4c-509a-4f8a-baed-249168b5d64e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:43.252517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42916a4c-509a-4f8a-baed-249168b5d64e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:43.252995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42916a4c-509a-4f8a-baed-249168b5d64e None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:43.253901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42916a4c-509a-4f8a-baed-249168b5d64e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:43.254415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42916a4c-509a-4f8a-baed-249168b5d64e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:43.255990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42916a4c-509a-4f8a-baed-249168b5d64e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:43.272093 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 359/725] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:43.278335 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35153664-974f-4359-915e-4dd362687c2c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:43.356678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178190-3ca0-4097-86ac-7dadebc0bc81 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:43.356869 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178190-3ca0-4097-86ac-7dadebc0bc81 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:43.357170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178190-3ca0-4097-86ac-7dadebc0bc81 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:43.358504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 360/726] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:33:43.368022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53e553-793f-4d70-97a0-d6eba02485af None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:43.368311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53e553-793f-4d70-97a0-d6eba02485af None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:43.368590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53e553-793f-4d70-97a0-d6eba02485af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:43.404955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ba53e553-793f-4d70-97a0-d6eba02485af None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:43.406178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ba53e553-793f-4d70-97a0-d6eba02485af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:43.441608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35153664-974f-4359-915e-4dd362687c2c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:43.455639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:43.503482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfc8f0-d827-4f68-a37b-88d65016144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:43.504766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfc8f0-d827-4f68-a37b-88d65016144b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:43.505208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfc8f0-d827-4f68-a37b-88d65016144b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:43.505618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfc8f0-d827-4f68-a37b-88d65016144b None admin] PATH_INFO: `/v3/users/swift` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:43.518066 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfc8f0-d827-4f68-a37b-88d65016144b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:43.521700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfc8f0-d827-4f68-a37b-88d65016144b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:43.524792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 361/727] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-59cfc8f0-d827-4f68-a37b-88d65016144b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:43.544208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:43.545386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:43.546536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:43.546536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:43.546536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:43.546536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:43.546536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:43.546536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:43.546536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:43.546536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:43.546536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:43.546536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:43.548647 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 367/728] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:43.554328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53e553-793f-4d70-97a0-d6eba02485af None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:43.690989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53e553-793f-4d70-97a0-d6eba02485af None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:43.732573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e176a71-2769-46f9-ba87-beb619986569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:43.733222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e176a71-2769-46f9-ba87-beb619986569 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:43.733509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e176a71-2769-46f9-ba87-beb619986569 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:43.733784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e176a71-2769-46f9-ba87-beb619986569 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:43.734520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e176a71-2769-46f9-ba87-beb619986569 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:43.734832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e176a71-2769-46f9-ba87-beb619986569 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:43.736271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e176a71-2769-46f9-ba87-beb619986569 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:43.765185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 362/729] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 18:33:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:43.767016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfc8f0-d827-4f68-a37b-88d65016144b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:43.811780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:43.870111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e176a71-2769-46f9-ba87-beb619986569 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:43.917142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfc8f0-d827-4f68-a37b-88d65016144b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:43.920816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:43.998806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e176a71-2769-46f9-ba87-beb619986569 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:44.002190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48ffd9-6f77-4076-b9d9-597db331491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:44.003057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48ffd9-6f77-4076-b9d9-597db331491d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.003540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48ffd9-6f77-4076-b9d9-597db331491d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.004006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48ffd9-6f77-4076-b9d9-597db331491d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.043043 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97ee1-8ae2-4f17-a4dc-a9396dfca246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:44.043840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97ee1-8ae2-4f17-a4dc-a9396dfca246 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.044126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97ee1-8ae2-4f17-a4dc-a9396dfca246 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.044403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97ee1-8ae2-4f17-a4dc-a9396dfca246 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.045544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d97ee1-8ae2-4f17-a4dc-a9396dfca246 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:44.047197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d97ee1-8ae2-4f17-a4dc-a9396dfca246 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:44.089515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48ffd9-6f77-4076-b9d9-597db331491d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:44.091093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48ffd9-6f77-4076-b9d9-597db331491d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:44.116189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:44.148934 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 363/730] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:33:43 2022] POST /identity/v3/services => generated 251 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:33:44.150237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb87c5-6f73-48d2-9eb9-22bb4e86f7fd None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.150507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb87c5-6f73-48d2-9eb9-22bb4e86f7fd None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.150852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb87c5-6f73-48d2-9eb9-22bb4e86f7fd None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.156680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 364/731] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:44.158174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5654cc8-72fe-4bbd-babf-3fe242af8451 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.158460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5654cc8-72fe-4bbd-babf-3fe242af8451 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.158735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5654cc8-72fe-4bbd-babf-3fe242af8451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.181464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e5654cc8-72fe-4bbd-babf-3fe242af8451 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:44.182837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e5654cc8-72fe-4bbd-babf-3fe242af8451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:44.211755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 368/732] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 18:33:44.221071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48ffd9-6f77-4076-b9d9-597db331491d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:44.392556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48ffd9-6f77-4076-b9d9-597db331491d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:44.419076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:44.446394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 365/733] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:44.448751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2d23-42dc-4339-9469-53ba6f2c274a None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.449217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2d23-42dc-4339-9469-53ba6f2c274a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.449750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2d23-42dc-4339-9469-53ba6f2c274a None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.450294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a23e05-69b0-4ba9-8a79-34b6c9d7e32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:44.451035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a23e05-69b0-4ba9-8a79-34b6c9d7e32c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.451412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a23e05-69b0-4ba9-8a79-34b6c9d7e32c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.451791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a23e05-69b0-4ba9-8a79-34b6c9d7e32c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.452156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 366/734] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:44.458375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc7f15-ba3d-4b82-9a15-5d00e2e26a02 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.459087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc7f15-ba3d-4b82-9a15-5d00e2e26a02 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.460038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc7f15-ba3d-4b82-9a15-5d00e2e26a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.469762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a23e05-69b0-4ba9-8a79-34b6c9d7e32c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:44.471493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a23e05-69b0-4ba9-8a79-34b6c9d7e32c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:44.481571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-5abc7f15-ba3d-4b82-9a15-5d00e2e26a02 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:44.482751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5abc7f15-ba3d-4b82-9a15-5d00e2e26a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-53a23e05-69b0-4ba9-8a79-34b6c9d7e32c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:44.488526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:44.489675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:44.490580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:44.490580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:44.490580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:44.492301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:44.497052 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 369/735] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:44.498927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a23e05-69b0-4ba9-8a79-34b6c9d7e32c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:44.560408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 367/736] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 18:33:44.612153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a23e05-69b0-4ba9-8a79-34b6c9d7e32c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:44.649382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb2388-8f95-472b-8337-acd8f9ac30c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:44.650000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb2388-8f95-472b-8337-acd8f9ac30c4 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.650280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb2388-8f95-472b-8337-acd8f9ac30c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.651517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb2388-8f95-472b-8337-acd8f9ac30c4 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.651627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627a4c-2ea1-45b7-92d5-8c30befbeca2 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.651732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb2388-8f95-472b-8337-acd8f9ac30c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:44.651809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627a4c-2ea1-45b7-92d5-8c30befbeca2 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.652751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb2388-8f95-472b-8337-acd8f9ac30c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:44.653415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627a4c-2ea1-45b7-92d5-8c30befbeca2 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb2388-8f95-472b-8337-acd8f9ac30c4 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:44.655303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 368/737] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:44.660322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aab752-2d29-457f-917c-330aa3bb7c97 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.660610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aab752-2d29-457f-917c-330aa3bb7c97 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.660890 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aab752-2d29-457f-917c-330aa3bb7c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.670891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 370/738] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:44.672107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb2388-8f95-472b-8337-acd8f9ac30c4 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:44.680762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f2aab752-2d29-457f-917c-330aa3bb7c97 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:44.682022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f2aab752-2d29-457f-917c-330aa3bb7c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:44.705412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:44.777680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb2388-8f95-472b-8337-acd8f9ac30c4 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:44.808358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 369/739] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 18:33:44.813642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aab752-2d29-457f-917c-330aa3bb7c97 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:44.830939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c30af6-5115-460c-9195-9a7ae37c58b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:44.831588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30af6-5115-460c-9195-9a7ae37c58b5 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:44.831869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30af6-5115-460c-9195-9a7ae37c58b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:44.832144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30af6-5115-460c-9195-9a7ae37c58b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:44.845383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c30af6-5115-460c-9195-9a7ae37c58b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:44.846840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c30af6-5115-460c-9195-9a7ae37c58b5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-a2c30af6-5115-460c-9195-9a7ae37c58b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:44.859634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:44.860560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:44.861506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:44.861506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:44.861506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:44.861506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:44.861506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:44.861506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:44.861506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:44.861506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:44.867616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 371/740] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:44.867616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c30af6-5115-460c-9195-9a7ae37c58b5 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:44.950249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:44.994368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aab752-2d29-457f-917c-330aa3bb7c97 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:45.001286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:45.015849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c30af6-5115-460c-9195-9a7ae37c58b5 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:45.040125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09058a-368e-4f1e-8564-0b0d8692a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:45.040926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09058a-368e-4f1e-8564-0b0d8692a187 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:45.041610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09058a-368e-4f1e-8564-0b0d8692a187 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:45.041997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09058a-368e-4f1e-8564-0b0d8692a187 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:45.055356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09058a-368e-4f1e-8564-0b0d8692a187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-2c09058a-368e-4f1e-8564-0b0d8692a187 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:45.060912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:45.061991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:45.062840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:45.063488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 370/741] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:45.064129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b7c4f-4102-4bb5-b228-aafe371d1597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:45.064651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09058a-368e-4f1e-8564-0b0d8692a187 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:45.065211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b7c4f-4102-4bb5-b228-aafe371d1597 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:45.065667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b7c4f-4102-4bb5-b228-aafe371d1597 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:45.066137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b7c4f-4102-4bb5-b228-aafe371d1597 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:45.067070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b7c4f-4102-4bb5-b228-aafe371d1597 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:45.067850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b7c4f-4102-4bb5-b228-aafe371d1597 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:45.070604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b7c4f-4102-4bb5-b228-aafe371d1597 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:45.087780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 372/742] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:45.088910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b7c4f-4102-4bb5-b228-aafe371d1597 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:45.186361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09058a-368e-4f1e-8564-0b0d8692a187 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:45.206022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b7c4f-4102-4bb5-b228-aafe371d1597 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:45.218520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:45.223408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df137cfe-6e66-4c47-8d9f-630e4140f17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:45.224018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df137cfe-6e66-4c47-8d9f-630e4140f17a None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:45.224301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df137cfe-6e66-4c47-8d9f-630e4140f17a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:45.224578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df137cfe-6e66-4c47-8d9f-630e4140f17a None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:45.225285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df137cfe-6e66-4c47-8d9f-630e4140f17a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:45.227041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df137cfe-6e66-4c47-8d9f-630e4140f17a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:45.244620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94a76c-47ba-4491-956b-7e4864737dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:45.245254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94a76c-47ba-4491-956b-7e4864737dab None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:45.245533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94a76c-47ba-4491-956b-7e4864737dab None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:45.245801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94a76c-47ba-4491-956b-7e4864737dab None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:45.246460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94a76c-47ba-4491-956b-7e4864737dab None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:45.246763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94a76c-47ba-4491-956b-7e4864737dab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:45.248101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94a76c-47ba-4491-956b-7e4864737dab None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:45.256004 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 373/743] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:45.257428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94a76c-47ba-4491-956b-7e4864737dab None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:45.272022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:45.341688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 371/744] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:33:45 2022] POST /identity/v3/services => generated 240 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:33:45.342702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df137cfe-6e66-4c47-8d9f-630e4140f17a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:45.369931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94a76c-47ba-4491-956b-7e4864737dab None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:45.425550 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9fc2a-024c-428e-aa9b-2f8e0fa10ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:45.426861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9fc2a-024c-428e-aa9b-2f8e0fa10ae3 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:45.427212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9fc2a-024c-428e-aa9b-2f8e0fa10ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:45.427761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9fc2a-024c-428e-aa9b-2f8e0fa10ae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:45.429206 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9fc2a-024c-428e-aa9b-2f8e0fa10ae3 None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=88a9b2cee21f4831b221d113ddbbd29d) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:45.429902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9fc2a-024c-428e-aa9b-2f8e0fa10ae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:45.432134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9fc2a-024c-428e-aa9b-2f8e0fa10ae3 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:45.440961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 374/745] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:45 2022] GET /identity/v3/role_assignments?user.id=88a9b2cee21f4831b221d113ddbbd29d&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=5c777de40b8449329d714d8c2229115e => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:45.441889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9fc2a-024c-428e-aa9b-2f8e0fa10ae3 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:45.512280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df137cfe-6e66-4c47-8d9f-630e4140f17a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:45.544538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:45.577598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae1d0f-853d-45c0-89f4-41c5d20a9f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:45.579679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae1d0f-853d-45c0-89f4-41c5d20a9f60 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:45.580225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae1d0f-853d-45c0-89f4-41c5d20a9f60 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:45.580726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae1d0f-853d-45c0-89f4-41c5d20a9f60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:45.609379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9fc2a-024c-428e-aa9b-2f8e0fa10ae3 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:45.612930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae1d0f-853d-45c0-89f4-41c5d20a9f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-d5ae1d0f-853d-45c0-89f4-41c5d20a9f60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:45.618891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:45.619891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:45.620747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:45.623550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 372/746] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:45.634163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae1d0f-853d-45c0-89f4-41c5d20a9f60 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:45.656673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10818c9-3978-4e30-a489-c27f3d034743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:45.658635 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10818c9-3978-4e30-a489-c27f3d034743 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:45.659129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10818c9-3978-4e30-a489-c27f3d034743 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:45.659626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10818c9-3978-4e30-a489-c27f3d034743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:45.673657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10818c9-3978-4e30-a489-c27f3d034743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:45.675730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10818c9-3978-4e30-a489-c27f3d034743 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-c10818c9-3978-4e30-a489-c27f3d034743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:45.696366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:45.697436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:45.698440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:45.698440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:45.698440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:45.700655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 375/747] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:45.708634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10818c9-3978-4e30-a489-c27f3d034743 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:45.712072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:45.764747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae1d0f-853d-45c0-89f4-41c5d20a9f60 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:45.813412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2ac21-38f7-4438-9138-287b7cc2870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:45.816087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ac21-38f7-4438-9138-287b7cc2870a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:45.816373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ac21-38f7-4438-9138-287b7cc2870a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:45.816650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ac21-38f7-4438-9138-287b7cc2870a None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:45.817369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2ac21-38f7-4438-9138-287b7cc2870a None admin] RBAC: Adding query filter params (name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:45.817679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2ac21-38f7-4438-9138-287b7cc2870a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:45.821146 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2ac21-38f7-4438-9138-287b7cc2870a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:45.840136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 373/748] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:45.847473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2ac21-38f7-4438-9138-287b7cc2870a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:45.887418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10818c9-3978-4e30-a489-c27f3d034743 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:45.917661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:45.931300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcecfe-7621-4582-b95c-39a53df1de5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:45.932353 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcecfe-7621-4582-b95c-39a53df1de5f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:45.932886 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcecfe-7621-4582-b95c-39a53df1de5f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:45.933436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcecfe-7621-4582-b95c-39a53df1de5f None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:45.934600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bcecfe-7621-4582-b95c-39a53df1de5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:45.935125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bcecfe-7621-4582-b95c-39a53df1de5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:45.937380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bcecfe-7621-4582-b95c-39a53df1de5f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:45.952084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 376/749] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:45.958562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcecfe-7621-4582-b95c-39a53df1de5f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:46.044344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2ac21-38f7-4438-9138-287b7cc2870a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:46.068261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:46.083609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53b68a-2a5d-4564-9fc0-ee485ed89d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:46.085303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53b68a-2a5d-4564-9fc0-ee485ed89d63 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:46.085804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53b68a-2a5d-4564-9fc0-ee485ed89d63 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:46.086393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53b68a-2a5d-4564-9fc0-ee485ed89d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:46.109842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53b68a-2a5d-4564-9fc0-ee485ed89d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:46.111318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53b68a-2a5d-4564-9fc0-ee485ed89d63 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-5c53b68a-2a5d-4564-9fc0-ee485ed89d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:46.139001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:46.140021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:46.140909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:46.140909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:46.140909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:46.140909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 374/750] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:46.151904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53b68a-2a5d-4564-9fc0-ee485ed89d63 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:46.213533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcecfe-7621-4582-b95c-39a53df1de5f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:46.252581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557cd41-22d8-4d4e-818c-5c6e84a8b276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:46.253319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557cd41-22d8-4d4e-818c-5c6e84a8b276 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:46.253602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557cd41-22d8-4d4e-818c-5c6e84a8b276 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:46.253928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557cd41-22d8-4d4e-818c-5c6e84a8b276 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:46.263634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557cd41-22d8-4d4e-818c-5c6e84a8b276 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:46.263634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557cd41-22d8-4d4e-818c-5c6e84a8b276 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-b557cd41-22d8-4d4e-818c-5c6e84a8b276 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:46.277717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:46.278922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:46.280117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:46.280117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:46.280117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:46.280117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:46.280117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:46.280117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:46.280117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:46.280117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:46.280117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:46.280117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:46.282027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 377/751] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 18:33:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:46.288905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557cd41-22d8-4d4e-818c-5c6e84a8b276 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:46.339776 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:46.347296 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53b68a-2a5d-4564-9fc0-ee485ed89d63 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:46.402828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557cd41-22d8-4d4e-818c-5c6e84a8b276 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:46.434202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45019099-bb5c-4d51-b5c5-2bb24fe74368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:46.434876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019099-bb5c-4d51-b5c5-2bb24fe74368 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:46.435163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019099-bb5c-4d51-b5c5-2bb24fe74368 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:46.435438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019099-bb5c-4d51-b5c5-2bb24fe74368 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:46.436196 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45019099-bb5c-4d51-b5c5-2bb24fe74368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:46.436507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45019099-bb5c-4d51-b5c5-2bb24fe74368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:46.439386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45019099-bb5c-4d51-b5c5-2bb24fe74368 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:46.453732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16c126-a346-4c35-8482-8de9c7a59568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:46.454857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16c126-a346-4c35-8482-8de9c7a59568 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:46.455146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16c126-a346-4c35-8482-8de9c7a59568 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:46.455425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16c126-a346-4c35-8482-8de9c7a59568 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:46.456229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16c126-a346-4c35-8482-8de9c7a59568 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:46.456604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16c126-a346-4c35-8482-8de9c7a59568 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:46.458057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16c126-a346-4c35-8482-8de9c7a59568 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:46.462670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 375/752] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:46.468027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45019099-bb5c-4d51-b5c5-2bb24fe74368 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:46.480264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 378/753] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 18:33:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:46.486151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16c126-a346-4c35-8482-8de9c7a59568 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:46.502848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:46.617870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45019099-bb5c-4d51-b5c5-2bb24fe74368 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:46.635796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16c126-a346-4c35-8482-8de9c7a59568 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:46.638859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:46.659470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebd9fb-5163-4bda-bf71-5f32fa71ae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:46.660570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebd9fb-5163-4bda-bf71-5f32fa71ae92 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:46.661432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebd9fb-5163-4bda-bf71-5f32fa71ae92 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:46.661916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebd9fb-5163-4bda-bf71-5f32fa71ae92 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:46.671018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebd9fb-5163-4bda-bf71-5f32fa71ae92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:46.673239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebd9fb-5163-4bda-bf71-5f32fa71ae92 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-20ebd9fb-5163-4bda-bf71-5f32fa71ae92 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:46.687645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:46.688648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:46.689625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:46.689625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:46.689625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:46.689625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:46.689625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:46.689625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:46.689625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:46.689625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:46.689625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:33:46.689625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:46.692369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 376/754] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 18:33:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:46.700158 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebd9fb-5163-4bda-bf71-5f32fa71ae92 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:46.720004 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c720f-675d-4b49-88fc-6c55c8eb8b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:46.721404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c720f-675d-4b49-88fc-6c55c8eb8b5e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:46.721890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c720f-675d-4b49-88fc-6c55c8eb8b5e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:46.722436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c720f-675d-4b49-88fc-6c55c8eb8b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:46.734331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c720f-675d-4b49-88fc-6c55c8eb8b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:46.736135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c720f-675d-4b49-88fc-6c55c8eb8b5e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-a03c720f-675d-4b49-88fc-6c55c8eb8b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:46.752413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:46.753426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:46.754417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:46.754417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:46.754417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:46.756972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 379/755] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:46.763214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c720f-675d-4b49-88fc-6c55c8eb8b5e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:46.878832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c720f-675d-4b49-88fc-6c55c8eb8b5e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:46.920477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34f63a-873c-40b0-bc4e-257190c78b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:46.921549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f63a-873c-40b0-bc4e-257190c78b61 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:46.922056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f63a-873c-40b0-bc4e-257190c78b61 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:46.922785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f63a-873c-40b0-bc4e-257190c78b61 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:46.923717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34f63a-873c-40b0-bc4e-257190c78b61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:46.924279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34f63a-873c-40b0-bc4e-257190c78b61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:46.925916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34f63a-873c-40b0-bc4e-257190c78b61 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:46.940858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 380/756] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:46.946687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:46.949130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34f63a-873c-40b0-bc4e-257190c78b61 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:46.959421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebd9fb-5163-4bda-bf71-5f32fa71ae92 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:46.966568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:47.039331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56f985-452e-47c9-9635-5b891207b396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:47.039957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f985-452e-47c9-9635-5b891207b396 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:47.040240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f985-452e-47c9-9635-5b891207b396 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:47.040517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f985-452e-47c9-9635-5b891207b396 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:47.041284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f985-452e-47c9-9635-5b891207b396 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:47.041598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f985-452e-47c9-9635-5b891207b396 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:47.043038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f985-452e-47c9-9635-5b891207b396 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:47.070150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 377/757] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 18:33:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:47.071389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34f63a-873c-40b0-bc4e-257190c78b61 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:47.076133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ab149-1bf7-467b-9cd8-850993f667e1 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:47.076133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ab149-1bf7-467b-9cd8-850993f667e1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:47.076133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ab149-1bf7-467b-9cd8-850993f667e1 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:47.076133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 378/758] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:47.077425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ab149-1bf7-467b-9cd8-850993f667e1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:47.117812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d840b2-58f3-41c0-b53d-9ee0441dfafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:47.118838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d840b2-58f3-41c0-b53d-9ee0441dfafa None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:47.119321 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d840b2-58f3-41c0-b53d-9ee0441dfafa None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:47.119826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d840b2-58f3-41c0-b53d-9ee0441dfafa None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:47.127734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d840b2-58f3-41c0-b53d-9ee0441dfafa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:47.129628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d840b2-58f3-41c0-b53d-9ee0441dfafa None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-98d840b2-58f3-41c0-b53d-9ee0441dfafa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:47.145443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:47.146369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:47.147940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:47.147940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:47.147940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:47.147940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:47.147940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:47.147940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:47.147940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:47.147940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:47.150845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 381/759] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:47.156718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05492444-c1e9-4107-9c5d-86d203154d2d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:47.157259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05492444-c1e9-4107-9c5d-86d203154d2d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:47.157761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05492444-c1e9-4107-9c5d-86d203154d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:47.177727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-05492444-c1e9-4107-9c5d-86d203154d2d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:47.180336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-05492444-c1e9-4107-9c5d-86d203154d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:47.186880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ab149-1bf7-467b-9cd8-850993f667e1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:47.223265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c46b-a8a3-4f75-ba3a-f659d06a1b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:47.223710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c46b-a8a3-4f75-ba3a-f659d06a1b36 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:47.223995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c46b-a8a3-4f75-ba3a-f659d06a1b36 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:47.224274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c46b-a8a3-4f75-ba3a-f659d06a1b36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:47.240240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:47.263201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 382/760] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:33:47.264412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05492444-c1e9-4107-9c5d-86d203154d2d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:47.267842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:47.316310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21c46b-a8a3-4f75-ba3a-f659d06a1b36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:47.317800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21c46b-a8a3-4f75-ba3a-f659d06a1b36 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:47.387451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05492444-c1e9-4107-9c5d-86d203154d2d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:47.425217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d61fe-0088-4998-bd78-854ad16c1558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:47.426385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d61fe-0088-4998-bd78-854ad16c1558 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:47.426870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d61fe-0088-4998-bd78-854ad16c1558 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:47.427345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d61fe-0088-4998-bd78-854ad16c1558 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:47.428250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d61fe-0088-4998-bd78-854ad16c1558 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:47.429248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d61fe-0088-4998-bd78-854ad16c1558 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:47.431125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d61fe-0088-4998-bd78-854ad16c1558 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:47.448327 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 383/761] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:47.454604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d61fe-0088-4998-bd78-854ad16c1558 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:47.505070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 379/762] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 18:33:47.514341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c46b-a8a3-4f75-ba3a-f659d06a1b36 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:47.537488 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:47.577652 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d61fe-0088-4998-bd78-854ad16c1558 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:47.607468 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:47.621912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3eb0b-a572-4f51-b885-2dbda2c02c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:47.622943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3eb0b-a572-4f51-b885-2dbda2c02c52 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:47.623508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3eb0b-a572-4f51-b885-2dbda2c02c52 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:47.623954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3eb0b-a572-4f51-b885-2dbda2c02c52 None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/1b29c5745a8e42ff8232e5ca22b4a4ad/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:47.626077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3eb0b-a572-4f51-b885-2dbda2c02c52 None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=1b29c5745a8e42ff8232e5ca22b4a4ad, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:47.627753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3eb0b-a572-4f51-b885-2dbda2c02c52 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:47.669365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c46b-a8a3-4f75-ba3a-f659d06a1b36 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:47.711305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 384/763] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:33:47 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/1b29c5745a8e42ff8232e5ca22b4a4ad/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:47.711510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9630b-fc4c-4b72-a359-475d18771357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:47.711510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9630b-fc4c-4b72-a359-475d18771357 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:47.711510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9630b-fc4c-4b72-a359-475d18771357 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:47.711510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9630b-fc4c-4b72-a359-475d18771357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:47.713916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3eb0b-a572-4f51-b885-2dbda2c02c52 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:47.724079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9630b-fc4c-4b72-a359-475d18771357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:47.726137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9630b-fc4c-4b72-a359-475d18771357 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-cae9630b-fc4c-4b72-a359-475d18771357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:47.748136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:47.749193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:47.750135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:47.750135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:47.750135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:47.750135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 380/764] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:47.757230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9630b-fc4c-4b72-a359-475d18771357 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:47.922804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3eb0b-a572-4f51-b885-2dbda2c02c52 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:47.952925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:47.966630 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e90c9-d6d2-4eff-b498-f0d3e2602ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:47.967165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:47.968629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e90c9-d6d2-4eff-b498-f0d3e2602ef3 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:47.970340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e90c9-d6d2-4eff-b498-f0d3e2602ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:47.970340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e90c9-d6d2-4eff-b498-f0d3e2602ef3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:47.970340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e90c9-d6d2-4eff-b498-f0d3e2602ef3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:47.971959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e90c9-d6d2-4eff-b498-f0d3e2602ef3 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:47.974727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9630b-fc4c-4b72-a359-475d18771357 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-ba4e90c9-d6d2-4eff-b498-f0d3e2602ef3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:47.982869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:47.984164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:47.985304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 385/765] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 8 18:33:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:47.989071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4113f79-a177-46a3-b074-b9d869f1500d None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:47.989071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4113f79-a177-46a3-b074-b9d869f1500d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:47.989071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4113f79-a177-46a3-b074-b9d869f1500d None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:47.989631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 386/766] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:47.994156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4113f79-a177-46a3-b074-b9d869f1500d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:48.020234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02e008-c2ee-43fd-b507-aa5f1276357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:48.021129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02e008-c2ee-43fd-b507-aa5f1276357a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.021762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02e008-c2ee-43fd-b507-aa5f1276357a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.022084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02e008-c2ee-43fd-b507-aa5f1276357a None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.023673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02e008-c2ee-43fd-b507-aa5f1276357a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:48.024197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02e008-c2ee-43fd-b507-aa5f1276357a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:48.026231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02e008-c2ee-43fd-b507-aa5f1276357a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:48.041590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 381/767] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:48.043354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaefe9-5b73-4438-8862-64ed916cc80f None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.044212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaefe9-5b73-4438-8862-64ed916cc80f None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.044494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcaefe9-5b73-4438-8862-64ed916cc80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.070263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-0fcaefe9-5b73-4438-8862-64ed916cc80f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:48.071936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0fcaefe9-5b73-4438-8862-64ed916cc80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:48.112134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4113f79-a177-46a3-b074-b9d869f1500d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:48.150826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 382/768] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:33:48.151510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcaefe9-5b73-4438-8862-64ed916cc80f None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:48.156122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa9dcc-1287-4e8e-b7b8-ea9a63b8efea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:48.156991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa9dcc-1287-4e8e-b7b8-ea9a63b8efea None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.157458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa9dcc-1287-4e8e-b7b8-ea9a63b8efea None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.157865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa9dcc-1287-4e8e-b7b8-ea9a63b8efea None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.158618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa9dcc-1287-4e8e-b7b8-ea9a63b8efea None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:48.159059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa9dcc-1287-4e8e-b7b8-ea9a63b8efea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:48.160614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa9dcc-1287-4e8e-b7b8-ea9a63b8efea None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:48.171546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 387/769] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 18:33:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:48.173960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399b063-f3c3-4dad-89d9-1161f4417a68 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.174395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399b063-f3c3-4dad-89d9-1161f4417a68 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.174916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399b063-f3c3-4dad-89d9-1161f4417a68 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.176710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 388/770] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:48.177620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399b063-f3c3-4dad-89d9-1161f4417a68 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:48.247099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:48.252880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:48.271460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcaefe9-5b73-4438-8862-64ed916cc80f None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:48.307914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399b063-f3c3-4dad-89d9-1161f4417a68 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:48.318819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f848e22-2c7f-4ab3-ae55-cf25f0f4df1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:48.320229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f848e22-2c7f-4ab3-ae55-cf25f0f4df1e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.320655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f848e22-2c7f-4ab3-ae55-cf25f0f4df1e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.321097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f848e22-2c7f-4ab3-ae55-cf25f0f4df1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.331010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f848e22-2c7f-4ab3-ae55-cf25f0f4df1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:48.332628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f848e22-2c7f-4ab3-ae55-cf25f0f4df1e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-0f848e22-2c7f-4ab3-ae55-cf25f0f4df1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:48.348148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:48.349040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 383/771] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24284b-8c65-468a-abec-3147f60fbad7 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.355231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24284b-8c65-468a-abec-3147f60fbad7 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.355871 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24284b-8c65-468a-abec-3147f60fbad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.359902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b7ffe-2211-45d0-b600-a24c6f92e5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:48.360516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b7ffe-2211-45d0-b600-a24c6f92e5db None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.360798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b7ffe-2211-45d0-b600-a24c6f92e5db None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.361120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b7ffe-2211-45d0-b600-a24c6f92e5db None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.361773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b7ffe-2211-45d0-b600-a24c6f92e5db None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:48.362068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b7ffe-2211-45d0-b600-a24c6f92e5db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:48.363497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b7ffe-2211-45d0-b600-a24c6f92e5db None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:48.374393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 389/772] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 18:33:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:48.374393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b7ffe-2211-45d0-b600-a24c6f92e5db None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:48.383396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-be24284b-8c65-468a-abec-3147f60fbad7 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:48.385448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-be24284b-8c65-468a-abec-3147f60fbad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:48.477155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 384/773] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:33:48.477155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24284b-8c65-468a-abec-3147f60fbad7 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:48.501880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b7ffe-2211-45d0-b600-a24c6f92e5db None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:48.523016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:48.546713 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fdadc-824f-4cd5-a1ff-e29b9c773d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:48.547440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fdadc-824f-4cd5-a1ff-e29b9c773d2b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.547785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fdadc-824f-4cd5-a1ff-e29b9c773d2b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.548130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fdadc-824f-4cd5-a1ff-e29b9c773d2b None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.548894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fdadc-824f-4cd5-a1ff-e29b9c773d2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:48.549384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fdadc-824f-4cd5-a1ff-e29b9c773d2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:48.550987 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fdadc-824f-4cd5-a1ff-e29b9c773d2b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:48.557970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:48.565947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 390/774] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:48.566806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fdadc-824f-4cd5-a1ff-e29b9c773d2b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:48.583920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24284b-8c65-468a-abec-3147f60fbad7 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:48.630334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2751e66-170c-45f5-aa0a-7ecbfad7a70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:48.631038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2751e66-170c-45f5-aa0a-7ecbfad7a70e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.631455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2751e66-170c-45f5-aa0a-7ecbfad7a70e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.631866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2751e66-170c-45f5-aa0a-7ecbfad7a70e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.633356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2751e66-170c-45f5-aa0a-7ecbfad7a70e None admin] RBAC: Adding query filter params (interface=public, service_id=0e4055d4eb1c41e59886b767adf1d7f5, region_id=RegionOne) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:48.633765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2751e66-170c-45f5-aa0a-7ecbfad7a70e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:48.635859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2751e66-170c-45f5-aa0a-7ecbfad7a70e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:48.647720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 385/775] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 18:33:48 2022] GET /identity/v3/endpoints?service_id=0e4055d4eb1c41e59886b767adf1d7f5&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:48.649003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2751e66-170c-45f5-aa0a-7ecbfad7a70e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:48.683500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fdadc-824f-4cd5-a1ff-e29b9c773d2b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:48.724602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb5c0f-dd44-4d6a-b9f1-feb0d90d98cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:48.725857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb5c0f-dd44-4d6a-b9f1-feb0d90d98cd None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.726143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb5c0f-dd44-4d6a-b9f1-feb0d90d98cd None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.726414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb5c0f-dd44-4d6a-b9f1-feb0d90d98cd None admin] PATH_INFO: `/v3/services/network` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.727027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb5c0f-dd44-4d6a-b9f1-feb0d90d98cd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:48.728493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb5c0f-dd44-4d6a-b9f1-feb0d90d98cd None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-97bb5c0f-dd44-4d6a-b9f1-feb0d90d98cd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:48.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:48.739427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 18:33:48.740282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:48.744574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 391/776] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:48 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:48.745529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb5c0f-dd44-4d6a-b9f1-feb0d90d98cd None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:48.784110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2751e66-170c-45f5-aa0a-7ecbfad7a70e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:48.790190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:48.826793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121853c-1033-4d3e-a794-0c3de3311167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:48.827402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121853c-1033-4d3e-a794-0c3de3311167 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.827681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121853c-1033-4d3e-a794-0c3de3311167 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.827952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121853c-1033-4d3e-a794-0c3de3311167 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.828671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121853c-1033-4d3e-a794-0c3de3311167 None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=3038d1d505654ffb95b78fbc9005baf4) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:48.828980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121853c-1033-4d3e-a794-0c3de3311167 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:48.830425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121853c-1033-4d3e-a794-0c3de3311167 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:48.841083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 386/777] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:48 2022] GET /identity/v3/role_assignments?user.id=3038d1d505654ffb95b78fbc9005baf4&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:48.841083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121853c-1033-4d3e-a794-0c3de3311167 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:48.846298 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:48.864165 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb5c0f-dd44-4d6a-b9f1-feb0d90d98cd None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:48.911235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb07fda-adf6-4c70-a0c4-68926859dac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:48.911891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb07fda-adf6-4c70-a0c4-68926859dac2 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:48.912218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb07fda-adf6-4c70-a0c4-68926859dac2 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:48.912500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb07fda-adf6-4c70-a0c4-68926859dac2 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:48.913688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb07fda-adf6-4c70-a0c4-68926859dac2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:48.915202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb07fda-adf6-4c70-a0c4-68926859dac2 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:48.921456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 392/778] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 18:33:48 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:48.922339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb07fda-adf6-4c70-a0c4-68926859dac2 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:48.964554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121853c-1033-4d3e-a794-0c3de3311167 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:49.018180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7861231-0704-462d-b77b-3fb5955831f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:49.019229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7861231-0704-462d-b77b-3fb5955831f8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:49.019652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7861231-0704-462d-b77b-3fb5955831f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:49.020068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7861231-0704-462d-b77b-3fb5955831f8 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:49.020729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7861231-0704-462d-b77b-3fb5955831f8 None admin] RBAC: Adding query filter params (name=network) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:49.021380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7861231-0704-462d-b77b-3fb5955831f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:49.023412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7861231-0704-462d-b77b-3fb5955831f8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:49.030075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 387/779] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:33:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:49.035896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7861231-0704-462d-b77b-3fb5955831f8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:49.047313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb07fda-adf6-4c70-a0c4-68926859dac2 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:49.051342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:49.093900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2b433-9b96-4bbe-94fd-b5a373fe72e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:49.096545 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2b433-9b96-4bbe-94fd-b5a373fe72e4 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:49.097595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2b433-9b96-4bbe-94fd-b5a373fe72e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:49.098903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2b433-9b96-4bbe-94fd-b5a373fe72e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:49.124795 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2b433-9b96-4bbe-94fd-b5a373fe72e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-f8f2b433-9b96-4bbe-94fd-b5a373fe72e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:49.131117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:49.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:49.133080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 393/780] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:49.140402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2b433-9b96-4bbe-94fd-b5a373fe72e4 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:49.166010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:49.205030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7861231-0704-462d-b77b-3fb5955831f8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:49.244748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa40385-304e-474e-a28c-8ff7f309b43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:49.245403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40385-304e-474e-a28c-8ff7f309b43e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:49.245681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40385-304e-474e-a28c-8ff7f309b43e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:49.245955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40385-304e-474e-a28c-8ff7f309b43e None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:49.246593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa40385-304e-474e-a28c-8ff7f309b43e None admin] RBAC: Adding query filter params (type=network) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:49.246893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa40385-304e-474e-a28c-8ff7f309b43e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:49.248264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa40385-304e-474e-a28c-8ff7f309b43e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:49.254240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 388/781] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:33:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:49.260264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa40385-304e-474e-a28c-8ff7f309b43e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:49.317172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2b433-9b96-4bbe-94fd-b5a373fe72e4 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:49.356143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff7c2f-49fa-4c47-9512-739937baa10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:49.356765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff7c2f-49fa-4c47-9512-739937baa10d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:49.357063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff7c2f-49fa-4c47-9512-739937baa10d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:49.357363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff7c2f-49fa-4c47-9512-739937baa10d None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:49.358033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff7c2f-49fa-4c47-9512-739937baa10d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:49.358343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff7c2f-49fa-4c47-9512-739937baa10d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:49.359710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff7c2f-49fa-4c47-9512-739937baa10d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:49.370228 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 394/782] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:49.376695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff7c2f-49fa-4c47-9512-739937baa10d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:49.411501 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:49.419137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa40385-304e-474e-a28c-8ff7f309b43e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:49.458405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:49.464288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40601013-1596-4250-bd88-46534b8bf531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:49.464914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40601013-1596-4250-bd88-46534b8bf531 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:49.465259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40601013-1596-4250-bd88-46534b8bf531 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:49.465548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40601013-1596-4250-bd88-46534b8bf531 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:49.466612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40601013-1596-4250-bd88-46534b8bf531 None admin] RBAC: Adding query filter params (interface=public, service_id=42063f68350946d498922383d1eb56b8, region_id=RegionOne) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:49.466930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40601013-1596-4250-bd88-46534b8bf531 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:49.468761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40601013-1596-4250-bd88-46534b8bf531 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:49.474977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 389/783] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 18:33:49 2022] GET /identity/v3/endpoints?service_id=42063f68350946d498922383d1eb56b8&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:49.481443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40601013-1596-4250-bd88-46534b8bf531 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:49.569583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff7c2f-49fa-4c47-9512-739937baa10d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:49.645709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e7908-46e3-4827-9d36-e72f6e4e638d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:49.646350 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e7908-46e3-4827-9d36-e72f6e4e638d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:49.646627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e7908-46e3-4827-9d36-e72f6e4e638d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:49.646901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e7908-46e3-4827-9d36-e72f6e4e638d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:49.659615 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40601013-1596-4250-bd88-46534b8bf531 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:49.672887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e7908-46e3-4827-9d36-e72f6e4e638d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:49.674370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e7908-46e3-4827-9d36-e72f6e4e638d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-d02e7908-46e3-4827-9d36-e72f6e4e638d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:49.700568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:49.701613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:49.702503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:49.702503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:49.702503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:49.702503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 395/784] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:49.707741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e7908-46e3-4827-9d36-e72f6e4e638d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:49.729027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ac99e-60cb-4520-8124-99665903d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:49.729667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ac99e-60cb-4520-8124-99665903d4fe None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:49.729945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ac99e-60cb-4520-8124-99665903d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:49.730217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ac99e-60cb-4520-8124-99665903d4fe None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:49.730828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ac99e-60cb-4520-8124-99665903d4fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:49.732199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ac99e-60cb-4520-8124-99665903d4fe None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:49.745473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 390/785] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 18:33:49 2022] GET /identity/v3/services => generated 772 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:49.873582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:49.887928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e7908-46e3-4827-9d36-e72f6e4e638d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:49.926964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a5126-1e69-4bbd-abcf-4fe94c5a587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:49.927579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a5126-1e69-4bbd-abcf-4fe94c5a587e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:49.927907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a5126-1e69-4bbd-abcf-4fe94c5a587e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:49.928194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a5126-1e69-4bbd-abcf-4fe94c5a587e None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:49.929334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a5126-1e69-4bbd-abcf-4fe94c5a587e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:49.929659 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a5126-1e69-4bbd-abcf-4fe94c5a587e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:49.931032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a5126-1e69-4bbd-abcf-4fe94c5a587e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:49.943664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 396/786] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:49.949710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ac99e-60cb-4520-8124-99665903d4fe None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:50.020944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:50.067755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ac99e-60cb-4520-8124-99665903d4fe None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:50.088768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d41d5-11e0-4476-8b62-a297c97a3aee None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:50.089317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d41d5-11e0-4476-8b62-a297c97a3aee None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:50.089395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d41d5-11e0-4476-8b62-a297c97a3aee None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:50.090852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 397/787] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:50.096681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825545b-cfda-4fb4-a45f-aa2dfceb58f1 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:50.097258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825545b-cfda-4fb4-a45f-aa2dfceb58f1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:50.097732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825545b-cfda-4fb4-a45f-aa2dfceb58f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:50.117016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-6825545b-cfda-4fb4-a45f-aa2dfceb58f1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:50.118727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6825545b-cfda-4fb4-a45f-aa2dfceb58f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:50.119672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934459f-35ea-4e30-8348-548a7a64b443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:50.120311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934459f-35ea-4e30-8348-548a7a64b443 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:50.120597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934459f-35ea-4e30-8348-548a7a64b443 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:50.120877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934459f-35ea-4e30-8348-548a7a64b443 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:50.137617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934459f-35ea-4e30-8348-548a7a64b443 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:50.142032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934459f-35ea-4e30-8348-548a7a64b443 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-1934459f-35ea-4e30-8348-548a7a64b443 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:50.170102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:50.171123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:50.172083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:50.172083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:50.172083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:50.172083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:50.172083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:50.172083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:50.172083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:50.172083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:50.172083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:50.172083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:50.177110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 391/788] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:50.180897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934459f-35ea-4e30-8348-548a7a64b443 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:50.249690 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 398/789] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:33:50.327598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934459f-35ea-4e30-8348-548a7a64b443 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:50.350741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:50.366655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d2d70-cd39-40fa-9aa1-019efec1692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:50.367523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d2d70-cd39-40fa-9aa1-019efec1692e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:50.367956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d2d70-cd39-40fa-9aa1-019efec1692e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:50.368377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d2d70-cd39-40fa-9aa1-019efec1692e None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:50.369342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d2d70-cd39-40fa-9aa1-019efec1692e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:50.369812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d2d70-cd39-40fa-9aa1-019efec1692e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:50.371475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d2d70-cd39-40fa-9aa1-019efec1692e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:50.390773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 392/790] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 18:33:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:50.397614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825545b-cfda-4fb4-a45f-aa2dfceb58f1 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:50.489672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:50.528658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825545b-cfda-4fb4-a45f-aa2dfceb58f1 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:50.577884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cbd3b-3546-40dd-a4b4-4977963ccf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:50.578516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cbd3b-3546-40dd-a4b4-4977963ccf0b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:50.578795 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cbd3b-3546-40dd-a4b4-4977963ccf0b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:50.579065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cbd3b-3546-40dd-a4b4-4977963ccf0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:50.602945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cbd3b-3546-40dd-a4b4-4977963ccf0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:50.603472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cbd3b-3546-40dd-a4b4-4977963ccf0b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-571cbd3b-3546-40dd-a4b4-4977963ccf0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:50.627126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:50.628156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:50.629120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:50.629120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:50.629120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:50.629315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 399/791] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:50.634971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d2d70-cd39-40fa-9aa1-019efec1692e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:50.639162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cbd3b-3546-40dd-a4b4-4977963ccf0b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:50.748864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cbd3b-3546-40dd-a4b4-4977963ccf0b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:50.781438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:50.790048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdec80-31cd-4978-b7fe-10c7dd939185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:50.791119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdec80-31cd-4978-b7fe-10c7dd939185 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:50.791609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdec80-31cd-4978-b7fe-10c7dd939185 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:50.792086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdec80-31cd-4978-b7fe-10c7dd939185 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:50.805639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdec80-31cd-4978-b7fe-10c7dd939185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-3cbdec80-31cd-4978-b7fe-10c7dd939185 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:50.811074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:50.812418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:50.813323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:50.816193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 400/792] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:50.822683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdec80-31cd-4978-b7fe-10c7dd939185 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:50.914748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d2d70-cd39-40fa-9aa1-019efec1692e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:50.940717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdec80-31cd-4978-b7fe-10c7dd939185 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:50.980500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174a306-9923-4ee1-ab0d-d57c0a4d3dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:50.981529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174a306-9923-4ee1-ab0d-d57c0a4d3dfa None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:50.982023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174a306-9923-4ee1-ab0d-d57c0a4d3dfa None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:50.982504 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174a306-9923-4ee1-ab0d-d57c0a4d3dfa None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:50.983381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174a306-9923-4ee1-ab0d-d57c0a4d3dfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:50.983919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174a306-9923-4ee1-ab0d-d57c0a4d3dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:50.985542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174a306-9923-4ee1-ab0d-d57c0a4d3dfa None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:50.995671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 401/793] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:51.001743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174a306-9923-4ee1-ab0d-d57c0a4d3dfa None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:51.024284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b99b4-0e79-4d64-b60a-777e978b506f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:51.024956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b99b4-0e79-4d64-b60a-777e978b506f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.025269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b99b4-0e79-4d64-b60a-777e978b506f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.025597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b99b4-0e79-4d64-b60a-777e978b506f None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.026257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b99b4-0e79-4d64-b60a-777e978b506f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:51.026606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b99b4-0e79-4d64-b60a-777e978b506f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:51.028270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b99b4-0e79-4d64-b60a-777e978b506f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:51.054163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 393/794] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:51.060011 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b99b4-0e79-4d64-b60a-777e978b506f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:51.061477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:51.145343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:51.187198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174a306-9923-4ee1-ab0d-d57c0a4d3dfa None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:51.223009 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b99b4-0e79-4d64-b60a-777e978b506f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:51.264150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2c693-b5b5-42ee-b1e7-711858e301e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:51.265156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2c693-b5b5-42ee-b1e7-711858e301e7 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.265789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2c693-b5b5-42ee-b1e7-711858e301e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.266292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2c693-b5b5-42ee-b1e7-711858e301e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.274433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2c693-b5b5-42ee-b1e7-711858e301e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:51.276186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2c693-b5b5-42ee-b1e7-711858e301e7 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-cec2c693-b5b5-42ee-b1e7-711858e301e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:51.289656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:51.290570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:51.291460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:51.291460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:51.291460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:51.291460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:51.291460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:51.291460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:51.291460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:51.291460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:51.291898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716cf0c-8220-44c1-8841-d196cc48756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:51.291898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716cf0c-8220-44c1-8841-d196cc48756e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.291898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716cf0c-8220-44c1-8841-d196cc48756e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.291898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716cf0c-8220-44c1-8841-d196cc48756e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.294300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 394/795] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:51.299615 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2c693-b5b5-42ee-b1e7-711858e301e7 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:51.309056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716cf0c-8220-44c1-8841-d196cc48756e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:51.310781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716cf0c-8220-44c1-8841-d196cc48756e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-9716cf0c-8220-44c1-8841-d196cc48756e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:51.326874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:51.327863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:51.328770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:51.328770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:51.328770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:51.330862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 402/796] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:51.337216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716cf0c-8220-44c1-8841-d196cc48756e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:51.450750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:51.458794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716cf0c-8220-44c1-8841-d196cc48756e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:51.501372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:51.503910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590d936-6e01-4c54-9547-d6a905925df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:51.504905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2c693-b5b5-42ee-b1e7-711858e301e7 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:51.506729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590d936-6e01-4c54-9547-d6a905925df5 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.508299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590d936-6e01-4c54-9547-d6a905925df5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.508800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590d936-6e01-4c54-9547-d6a905925df5 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.509757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590d936-6e01-4c54-9547-d6a905925df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:51.510267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590d936-6e01-4c54-9547-d6a905925df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:51.511881 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590d936-6e01-4c54-9547-d6a905925df5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:51.527875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 403/797] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:51.534478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa1f5-ca33-498f-b98a-40d2c0456c28 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.535003 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa1f5-ca33-498f-b98a-40d2c0456c28 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.535487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa1f5-ca33-498f-b98a-40d2c0456c28 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.537245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 404/798] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:51.538504 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafa1f5-ca33-498f-b98a-40d2c0456c28 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:51.546934 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeadd32-120b-4c90-bf4f-b1777d74725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:51.547556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeadd32-120b-4c90-bf4f-b1777d74725f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.547836 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeadd32-120b-4c90-bf4f-b1777d74725f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.548112 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeadd32-120b-4c90-bf4f-b1777d74725f None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.548809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeadd32-120b-4c90-bf4f-b1777d74725f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:51.549147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeadd32-120b-4c90-bf4f-b1777d74725f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:51.550586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeadd32-120b-4c90-bf4f-b1777d74725f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:51.563884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 395/799] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:51.569336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94657d84-0c2b-4ef3-9c72-4362438c8cc1 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.569794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94657d84-0c2b-4ef3-9c72-4362438c8cc1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.570184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94657d84-0c2b-4ef3-9c72-4362438c8cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.596193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-94657d84-0c2b-4ef3-9c72-4362438c8cc1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:51.597361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-94657d84-0c2b-4ef3-9c72-4362438c8cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:51.672271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafa1f5-ca33-498f-b98a-40d2c0456c28 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:51.676776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 396/800] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:33:51.677725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94657d84-0c2b-4ef3-9c72-4362438c8cc1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:51.714538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0310250-f507-4671-ad04-ed0d8d3ee270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:51.715584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0310250-f507-4671-ad04-ed0d8d3ee270 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.716103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0310250-f507-4671-ad04-ed0d8d3ee270 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.716581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0310250-f507-4671-ad04-ed0d8d3ee270 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.727596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0310250-f507-4671-ad04-ed0d8d3ee270 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:51.730213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0310250-f507-4671-ad04-ed0d8d3ee270 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-b0310250-f507-4671-ad04-ed0d8d3ee270 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:51.761707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:51.762750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:51.763717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:51.763717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:51.763717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:51.763717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:51.763717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:51.763717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:51.763717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:51.763717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:51.763717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:33:51.763717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:51.766009 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 405/801] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 18:33:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:51.777787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0310250-f507-4671-ad04-ed0d8d3ee270 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:51.799896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:51.836818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:51.858676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94657d84-0c2b-4ef3-9c72-4362438c8cc1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:51.909031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db264ef-f7a0-40ad-b501-eb965045c54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:51.909715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db264ef-f7a0-40ad-b501-eb965045c54f None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.909994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db264ef-f7a0-40ad-b501-eb965045c54f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.910267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db264ef-f7a0-40ad-b501-eb965045c54f None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/88a9b2cee21f4831b221d113ddbbd29d/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.912293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db264ef-f7a0-40ad-b501-eb965045c54f None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=88a9b2cee21f4831b221d113ddbbd29d, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:51.914007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db264ef-f7a0-40ad-b501-eb965045c54f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:51.923783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0310250-f507-4671-ad04-ed0d8d3ee270 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:51.963100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47cc89-5b8a-4ba3-aacb-9e8a8d56fd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:51.963715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47cc89-5b8a-4ba3-aacb-9e8a8d56fd5b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.963999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47cc89-5b8a-4ba3-aacb-9e8a8d56fd5b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.964274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47cc89-5b8a-4ba3-aacb-9e8a8d56fd5b None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.964998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47cc89-5b8a-4ba3-aacb-9e8a8d56fd5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:51.965338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47cc89-5b8a-4ba3-aacb-9e8a8d56fd5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:51.966758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47cc89-5b8a-4ba3-aacb-9e8a8d56fd5b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:51.985562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 406/802] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 18:33:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:51.986803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef7c1f-8be9-48ce-8dcc-978867601387 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:51.987093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef7c1f-8be9-48ce-8dcc-978867601387 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:51.987416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef7c1f-8be9-48ce-8dcc-978867601387 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:51.991103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 407/803] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:51.991103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef7c1f-8be9-48ce-8dcc-978867601387 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:52.013307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 397/804] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:33:51 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/88a9b2cee21f4831b221d113ddbbd29d/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:52.014868 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db264ef-f7a0-40ad-b501-eb965045c54f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:52.071943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:52.108349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef7c1f-8be9-48ce-8dcc-978867601387 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:52.148402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ab5bf-346b-468c-9610-b05817eb423c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:52.149277 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ab5bf-346b-468c-9610-b05817eb423c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:52.149629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ab5bf-346b-468c-9610-b05817eb423c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:52.150779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ab5bf-346b-468c-9610-b05817eb423c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:52.165594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ab5bf-346b-468c-9610-b05817eb423c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-040ab5bf-346b-468c-9610-b05817eb423c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:52.170661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:52.171696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:52.172556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 408/805] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:52.173328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abb5ea-cce9-4b70-8c67-fb159960fa5c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:52.173619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abb5ea-cce9-4b70-8c67-fb159960fa5c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:52.174231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abb5ea-cce9-4b70-8c67-fb159960fa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:52.179230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db264ef-f7a0-40ad-b501-eb965045c54f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:52.195312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-53abb5ea-cce9-4b70-8c67-fb159960fa5c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:52.196722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-53abb5ea-cce9-4b70-8c67-fb159960fa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:52.227520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd9dd6-b592-4037-bbd6-41a196324904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:52.228708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd9dd6-b592-4037-bbd6-41a196324904 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:52.229232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd9dd6-b592-4037-bbd6-41a196324904 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:52.229876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd9dd6-b592-4037-bbd6-41a196324904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:52.254264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:52.290373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 409/806] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:33:52.291247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abb5ea-cce9-4b70-8c67-fb159960fa5c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:52.339225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd9dd6-b592-4037-bbd6-41a196324904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:52.340614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd9dd6-b592-4037-bbd6-41a196324904 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:52.356475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:52.399123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abb5ea-cce9-4b70-8c67-fb159960fa5c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:52.437727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ee24d-60d4-47ba-a37a-88a75568ac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:52.438340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ee24d-60d4-47ba-a37a-88a75568ac2f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:52.438619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ee24d-60d4-47ba-a37a-88a75568ac2f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:52.438890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ee24d-60d4-47ba-a37a-88a75568ac2f None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:52.439554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ee24d-60d4-47ba-a37a-88a75568ac2f None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:52.439857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ee24d-60d4-47ba-a37a-88a75568ac2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:52.445068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ee24d-60d4-47ba-a37a-88a75568ac2f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:52.451094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 410/807] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:52.456609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ee24d-60d4-47ba-a37a-88a75568ac2f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:52.543865 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 398/808] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:33:52.550816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd9dd6-b592-4037-bbd6-41a196324904 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:52.627277 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ee24d-60d4-47ba-a37a-88a75568ac2f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:52.641503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:52.653313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:52.664386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13083bff-af12-4864-827e-61b2deefcd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:52.665021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13083bff-af12-4864-827e-61b2deefcd4d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:52.665341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13083bff-af12-4864-827e-61b2deefcd4d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:52.665619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13083bff-af12-4864-827e-61b2deefcd4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:52.676485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13083bff-af12-4864-827e-61b2deefcd4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:52.677976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13083bff-af12-4864-827e-61b2deefcd4d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:52.691195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd9dd6-b592-4037-bbd6-41a196324904 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-13083bff-af12-4864-827e-61b2deefcd4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:52.694416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:52.701313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:52.702242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:52.702242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:52.702242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:52.702242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 411/809] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:52.702242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13083bff-af12-4864-827e-61b2deefcd4d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:52.727245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460ca6b-8a50-4f7a-856d-8e4c18d08d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:52.727872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ca6b-8a50-4f7a-856d-8e4c18d08d23 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:52.728148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ca6b-8a50-4f7a-856d-8e4c18d08d23 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:52.728416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460ca6b-8a50-4f7a-856d-8e4c18d08d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:52.738938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460ca6b-8a50-4f7a-856d-8e4c18d08d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:52.740355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460ca6b-8a50-4f7a-856d-8e4c18d08d23 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-8460ca6b-8a50-4f7a-856d-8e4c18d08d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:52.761301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:52.762316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:52.768638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:52.768638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:52.768638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:52.768638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 399/810] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:52.768638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f0554-92f6-4242-8392-8e07f2baff36 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:52.768638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f0554-92f6-4242-8392-8e07f2baff36 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:52.768638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f0554-92f6-4242-8392-8e07f2baff36 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:52.768638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 400/811] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:52.770777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f0554-92f6-4242-8392-8e07f2baff36 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:52.822178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13083bff-af12-4864-827e-61b2deefcd4d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:52.859698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa93216-a6ae-4407-9a8e-c808fdd4e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:52.860245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa93216-a6ae-4407-9a8e-c808fdd4e87c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:52.860570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa93216-a6ae-4407-9a8e-c808fdd4e87c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:52.860855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa93216-a6ae-4407-9a8e-c808fdd4e87c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:52.861480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa93216-a6ae-4407-9a8e-c808fdd4e87c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:52.862835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa93216-a6ae-4407-9a8e-c808fdd4e87c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-afa93216-a6ae-4407-9a8e-c808fdd4e87c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:52.872796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:52.873983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 18:33:52.874888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:52.877398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:52.878888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 412/812] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 8 18:33:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:52.879743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa93216-a6ae-4407-9a8e-c808fdd4e87c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:52.881900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f0554-92f6-4242-8392-8e07f2baff36 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:52.919291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b6eff-1959-4ba6-910f-90ea0188ac0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:52.919899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b6eff-1959-4ba6-910f-90ea0188ac0a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:52.920175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b6eff-1959-4ba6-910f-90ea0188ac0a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:52.920451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b6eff-1959-4ba6-910f-90ea0188ac0a None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:52.921113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b6eff-1959-4ba6-910f-90ea0188ac0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:52.921419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b6eff-1959-4ba6-910f-90ea0188ac0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:52.922828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b6eff-1959-4ba6-910f-90ea0188ac0a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:52.927613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:52.940084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 401/813] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:52.940084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1947b42-2c87-4218-aa46-788dee444dc3 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:52.940084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1947b42-2c87-4218-aa46-788dee444dc3 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:52.940084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1947b42-2c87-4218-aa46-788dee444dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:52.955837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b1947b42-2c87-4218-aa46-788dee444dc3 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:52.957027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b1947b42-2c87-4218-aa46-788dee444dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:52.995449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa93216-a6ae-4407-9a8e-c808fdd4e87c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:53.032971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 402/814] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:33:53.033787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1947b42-2c87-4218-aa46-788dee444dc3 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:53.041716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012688c8-599f-4a09-ae53-0d9400ed8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:53.042604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012688c8-599f-4a09-ae53-0d9400ed8c5c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:53.042887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012688c8-599f-4a09-ae53-0d9400ed8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:53.043161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012688c8-599f-4a09-ae53-0d9400ed8c5c None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:53.043828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012688c8-599f-4a09-ae53-0d9400ed8c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:53.044133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012688c8-599f-4a09-ae53-0d9400ed8c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:53.045513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012688c8-599f-4a09-ae53-0d9400ed8c5c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:53.061199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 413/815] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:53.062345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012688c8-599f-4a09-ae53-0d9400ed8c5c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:53.161628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:53.175193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1947b42-2c87-4218-aa46-788dee444dc3 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:53.180227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:53.183655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012688c8-599f-4a09-ae53-0d9400ed8c5c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:53.213974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad54ad0-d7c9-4b9b-b978-d37ee318fc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:53.214603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54ad0-d7c9-4b9b-b978-d37ee318fc7d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:53.214870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54ad0-d7c9-4b9b-b978-d37ee318fc7d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:53.215145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad54ad0-d7c9-4b9b-b978-d37ee318fc7d None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:53.215776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad54ad0-d7c9-4b9b-b978-d37ee318fc7d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:53.216083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad54ad0-d7c9-4b9b-b978-d37ee318fc7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:53.217632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad54ad0-d7c9-4b9b-b978-d37ee318fc7d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:53.222913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 403/816] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 18:33:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:53.224205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad54ad0-d7c9-4b9b-b978-d37ee318fc7d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:53.227979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f111c4-1cd9-4e95-b5e7-271316de5a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:53.229373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f111c4-1cd9-4e95-b5e7-271316de5a0d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:53.229716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f111c4-1cd9-4e95-b5e7-271316de5a0d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:53.230080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f111c4-1cd9-4e95-b5e7-271316de5a0d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:53.239762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f111c4-1cd9-4e95-b5e7-271316de5a0d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:53.243354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f111c4-1cd9-4e95-b5e7-271316de5a0d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-11f111c4-1cd9-4e95-b5e7-271316de5a0d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:53.261389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:53.262402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:53.263351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 414/817] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 18:33:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:53.271706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f111c4-1cd9-4e95-b5e7-271316de5a0d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:53.363481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad54ad0-d7c9-4b9b-b978-d37ee318fc7d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:53.388479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f111c4-1cd9-4e95-b5e7-271316de5a0d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:53.401281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc299e79-a589-46a0-9afd-dd6987815eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:53.401912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc299e79-a589-46a0-9afd-dd6987815eff None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:53.402245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc299e79-a589-46a0-9afd-dd6987815eff None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:53.402522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc299e79-a589-46a0-9afd-dd6987815eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:53.409329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc299e79-a589-46a0-9afd-dd6987815eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:53.410769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc299e79-a589-46a0-9afd-dd6987815eff None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-bc299e79-a589-46a0-9afd-dd6987815eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:53.422239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:53.423134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:53.424039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:53.424039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:53.424039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:53.424039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:53.424039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:53.424039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:53.424039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:53.424039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:53.424039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 404/818] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:53.424039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc299e79-a589-46a0-9afd-dd6987815eff None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:53.433945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45613a39-07f8-4ad4-8dfd-4069db536af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:53.434558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45613a39-07f8-4ad4-8dfd-4069db536af6 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:53.434834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45613a39-07f8-4ad4-8dfd-4069db536af6 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:53.435102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45613a39-07f8-4ad4-8dfd-4069db536af6 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:53.435731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45613a39-07f8-4ad4-8dfd-4069db536af6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:53.436028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45613a39-07f8-4ad4-8dfd-4069db536af6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:53.437484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45613a39-07f8-4ad4-8dfd-4069db536af6 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:53.440450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:53.448056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 415/819] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 8 18:33:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:53.448056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45613a39-07f8-4ad4-8dfd-4069db536af6 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:53.470460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:53.536394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc299e79-a589-46a0-9afd-dd6987815eff None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:53.552029 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45613a39-07f8-4ad4-8dfd-4069db536af6 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:53.573705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6abaa-eaf1-4a87-8e55-0114fe1ba615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:53.574320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6abaa-eaf1-4a87-8e55-0114fe1ba615 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:53.574601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6abaa-eaf1-4a87-8e55-0114fe1ba615 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:53.574878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6abaa-eaf1-4a87-8e55-0114fe1ba615 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:53.575608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6abaa-eaf1-4a87-8e55-0114fe1ba615 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:53.575919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6abaa-eaf1-4a87-8e55-0114fe1ba615 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:53.577365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6abaa-eaf1-4a87-8e55-0114fe1ba615 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:53.593959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f32fbf-3315-4789-9162-d99e2de77671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:53.594846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f32fbf-3315-4789-9162-d99e2de77671 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:53.595299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f32fbf-3315-4789-9162-d99e2de77671 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:53.595628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 405/820] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 18:33:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:53.595928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f32fbf-3315-4789-9162-d99e2de77671 None admin] PATH_INFO: `/v3/services/network` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:53.596509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6abaa-eaf1-4a87-8e55-0114fe1ba615 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:53.597026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f32fbf-3315-4789-9162-d99e2de77671 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:53.598621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f32fbf-3315-4789-9162-d99e2de77671 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-89f32fbf-3315-4789-9162-d99e2de77671 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:53.614070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:53.615119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:53.616526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 416/821] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:53 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:53.617386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f32fbf-3315-4789-9162-d99e2de77671 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:53.678818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:53.711937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:53.714260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6abaa-eaf1-4a87-8e55-0114fe1ba615 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:53.723631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f32fbf-3315-4789-9162-d99e2de77671 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:53.758265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c75187-b26f-4baf-84e8-4bef791ea42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:53.758819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d42150-40ff-489b-8ff7-39a2fdb54fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:53.758943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c75187-b26f-4baf-84e8-4bef791ea42a None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:53.759130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c75187-b26f-4baf-84e8-4bef791ea42a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:53.759393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c75187-b26f-4baf-84e8-4bef791ea42a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:53.759493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d42150-40ff-489b-8ff7-39a2fdb54fa5 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:53.759696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d42150-40ff-489b-8ff7-39a2fdb54fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:53.759965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d42150-40ff-489b-8ff7-39a2fdb54fa5 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:53.760061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c75187-b26f-4baf-84e8-4bef791ea42a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:53.760651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d42150-40ff-489b-8ff7-39a2fdb54fa5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:53.760956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d42150-40ff-489b-8ff7-39a2fdb54fa5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:53.761409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c75187-b26f-4baf-84e8-4bef791ea42a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:53.762402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d42150-40ff-489b-8ff7-39a2fdb54fa5 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:53.777290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 406/822] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:53.778152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d42150-40ff-489b-8ff7-39a2fdb54fa5 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:53.868011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 417/823] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:33:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:33:53.868865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c75187-b26f-4baf-84e8-4bef791ea42a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:53.890793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d42150-40ff-489b-8ff7-39a2fdb54fa5 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:53.935973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:53.951559 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dabb63-48db-4383-a871-76f14632dae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:53.952373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabb63-48db-4383-a871-76f14632dae2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:53.955975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabb63-48db-4383-a871-76f14632dae2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:53.955975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabb63-48db-4383-a871-76f14632dae2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:53.969033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dabb63-48db-4383-a871-76f14632dae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:53.970558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dabb63-48db-4383-a871-76f14632dae2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-e5dabb63-48db-4383-a871-76f14632dae2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:53.992322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:53.993410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:53.994306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:53.994306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:53.994306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:54.000174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 407/824] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:54.001163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dabb63-48db-4383-a871-76f14632dae2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:54.012478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c75187-b26f-4baf-84e8-4bef791ea42a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:54.053780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c8a0f-70cb-421f-9a90-86297e71447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:54.054412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8a0f-70cb-421f-9a90-86297e71447f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:54.054694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8a0f-70cb-421f-9a90-86297e71447f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:54.054970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8a0f-70cb-421f-9a90-86297e71447f None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:54.055607 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8a0f-70cb-421f-9a90-86297e71447f None admin] RBAC: Adding query filter params (name=network) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:54.055911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8a0f-70cb-421f-9a90-86297e71447f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:54.057307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8a0f-70cb-421f-9a90-86297e71447f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:54.067808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 418/825] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:33:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:54.068576 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c8a0f-70cb-421f-9a90-86297e71447f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:54.080891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:54.133542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dabb63-48db-4383-a871-76f14632dae2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:54.176699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d88fd-683c-4063-a6b8-f9c87f59ce37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:54.178015 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d88fd-683c-4063-a6b8-f9c87f59ce37 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:54.179260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d88fd-683c-4063-a6b8-f9c87f59ce37 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:54.179681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d88fd-683c-4063-a6b8-f9c87f59ce37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:54.180680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d88fd-683c-4063-a6b8-f9c87f59ce37 None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=1b29c5745a8e42ff8232e5ca22b4a4ad) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:54.181182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d88fd-683c-4063-a6b8-f9c87f59ce37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:54.183482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d88fd-683c-4063-a6b8-f9c87f59ce37 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:54.192309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 408/826] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:54 2022] GET /identity/v3/role_assignments?user.id=1b29c5745a8e42ff8232e5ca22b4a4ad&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=5c777de40b8449329d714d8c2229115e => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:54.194241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d88fd-683c-4063-a6b8-f9c87f59ce37 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:54.210691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c8a0f-70cb-421f-9a90-86297e71447f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:54.262178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:54.331978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d88fd-683c-4063-a6b8-f9c87f59ce37 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:54.342566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb122cd-4f6d-4c9a-a937-6f56777a5b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:54.343205 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb122cd-4f6d-4c9a-a937-6f56777a5b3f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:54.343755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb122cd-4f6d-4c9a-a937-6f56777a5b3f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:54.344027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb122cd-4f6d-4c9a-a937-6f56777a5b3f None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:54.344688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb122cd-4f6d-4c9a-a937-6f56777a5b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:54.344996 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb122cd-4f6d-4c9a-a937-6f56777a5b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:54.346446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb122cd-4f6d-4c9a-a937-6f56777a5b3f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:54.360153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 419/827] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:54.361328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11175b6b-8123-4cec-90d7-dce6ab224219 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:54.361619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11175b6b-8123-4cec-90d7-dce6ab224219 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:54.361907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11175b6b-8123-4cec-90d7-dce6ab224219 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:54.368032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 420/828] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:54.368972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11175b6b-8123-4cec-90d7-dce6ab224219 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:54.381691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fccc3-3795-4346-879e-67b750e8614a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:54.383068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fccc3-3795-4346-879e-67b750e8614a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:54.383777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fccc3-3795-4346-879e-67b750e8614a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:54.384211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fccc3-3795-4346-879e-67b750e8614a None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:54.385016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fccc3-3795-4346-879e-67b750e8614a None admin] RBAC: Adding query filter params (type=network) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:54.385720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fccc3-3795-4346-879e-67b750e8614a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:54.388198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fccc3-3795-4346-879e-67b750e8614a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:54.396277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 409/829] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:33:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:54.399310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b192f-1190-4d42-a8f6-f236ef9fa149 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:54.400108 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b192f-1190-4d42-a8f6-f236ef9fa149 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:54.400613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b192f-1190-4d42-a8f6-f236ef9fa149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:54.426498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-8c5b192f-1190-4d42-a8f6-f236ef9fa149 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:54.428740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8c5b192f-1190-4d42-a8f6-f236ef9fa149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:54.451930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:54.497297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11175b6b-8123-4cec-90d7-dce6ab224219 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:54.526403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 410/830] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:33:54.527838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b192f-1190-4d42-a8f6-f236ef9fa149 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:54.546971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981b22-2eef-4f8d-9e3b-a8067388f596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:54.547619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981b22-2eef-4f8d-9e3b-a8067388f596 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:54.548631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981b22-2eef-4f8d-9e3b-a8067388f596 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:54.548631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981b22-2eef-4f8d-9e3b-a8067388f596 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:54.558753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37981b22-2eef-4f8d-9e3b-a8067388f596 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:54.560891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37981b22-2eef-4f8d-9e3b-a8067388f596 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:54.574991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-37981b22-2eef-4f8d-9e3b-a8067388f596 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:54.576707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:54.577858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:54.578763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:54.578763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:54.578763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:54.578763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:54.578763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:54.578763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:54.578763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:54.578763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:54.578763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 421/831] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:54.583670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981b22-2eef-4f8d-9e3b-a8067388f596 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:54.649313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b192f-1190-4d42-a8f6-f236ef9fa149 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:54.703831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3993bb0-c6a6-47b6-9a75-ff759417e3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:54.704813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3993bb0-c6a6-47b6-9a75-ff759417e3fc None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:54.705341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3993bb0-c6a6-47b6-9a75-ff759417e3fc None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:54.705818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3993bb0-c6a6-47b6-9a75-ff759417e3fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:54.706675 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3993bb0-c6a6-47b6-9a75-ff759417e3fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:54.708288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3993bb0-c6a6-47b6-9a75-ff759417e3fc None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:54.745378 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981b22-2eef-4f8d-9e3b-a8067388f596 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:54.775739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:54.785019 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889f56d-6a40-4521-88d9-419c8ea4d93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:54.786124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f56d-6a40-4521-88d9-419c8ea4d93c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:54.786590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f56d-6a40-4521-88d9-419c8ea4d93c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:54.787048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f56d-6a40-4521-88d9-419c8ea4d93c None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:54.788661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889f56d-6a40-4521-88d9-419c8ea4d93c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:54.789188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889f56d-6a40-4521-88d9-419c8ea4d93c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:54.791076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889f56d-6a40-4521-88d9-419c8ea4d93c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:54.808379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 411/832] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:33:54 2022] POST /identity/v3/endpoints => generated 337 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:33:54.811027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 422/833] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:54.817425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3993bb0-c6a6-47b6-9a75-ff759417e3fc None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:54.854621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889f56d-6a40-4521-88d9-419c8ea4d93c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:54.929232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:54.950182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3993bb0-c6a6-47b6-9a75-ff759417e3fc None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:54.978935 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889f56d-6a40-4521-88d9-419c8ea4d93c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:55.026951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690858c-41f7-4275-9b2a-b925175aaa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:55.027957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28b466-cfda-4d98-a29a-57e0282eca9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:55.028904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690858c-41f7-4275-9b2a-b925175aaa65 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:55.029842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28b466-cfda-4d98-a29a-57e0282eca9f None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:55.030336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690858c-41f7-4275-9b2a-b925175aaa65 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:55.030946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28b466-cfda-4d98-a29a-57e0282eca9f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:55.031366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690858c-41f7-4275-9b2a-b925175aaa65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:55.032062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28b466-cfda-4d98-a29a-57e0282eca9f None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/3038d1d505654ffb95b78fbc9005baf4/roles/72ffa7e7e6374ac395b22fe6f5723045` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:55.057097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690858c-41f7-4275-9b2a-b925175aaa65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-4690858c-41f7-4275-9b2a-b925175aaa65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:55.065203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:55.066232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:33:55.067084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:55.070209 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28b466-cfda-4d98-a29a-57e0282eca9f None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=3038d1d505654ffb95b78fbc9005baf4, role_id=72ffa7e7e6374ac395b22fe6f5723045)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:55.072031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28b466-cfda-4d98-a29a-57e0282eca9f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:55.076159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 423/834] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:55.082318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690858c-41f7-4275-9b2a-b925175aaa65 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:55.138166 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:55.156937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 412/835] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:33:54 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/3038d1d505654ffb95b78fbc9005baf4/roles/72ffa7e7e6374ac395b22fe6f5723045 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:33:55.203314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690858c-41f7-4275-9b2a-b925175aaa65 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:55.249944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976a9b2-85a1-4ea0-a449-bedb2132ecab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:55.250910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976a9b2-85a1-4ea0-a449-bedb2132ecab None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:55.251398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976a9b2-85a1-4ea0-a449-bedb2132ecab None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:55.251876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976a9b2-85a1-4ea0-a449-bedb2132ecab None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:55.252796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976a9b2-85a1-4ea0-a449-bedb2132ecab None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:55.253338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976a9b2-85a1-4ea0-a449-bedb2132ecab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:55.254945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976a9b2-85a1-4ea0-a449-bedb2132ecab None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:55.265450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 424/836] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:33:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:55.271694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28b466-cfda-4d98-a29a-57e0282eca9f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:55.403281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28b466-cfda-4d98-a29a-57e0282eca9f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:55.439948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec3a9e-83e7-4c25-bec1-090ef0f82212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:55.440927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec3a9e-83e7-4c25-bec1-090ef0f82212 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:55.441437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec3a9e-83e7-4c25-bec1-090ef0f82212 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:55.441945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec3a9e-83e7-4c25-bec1-090ef0f82212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:55.452055 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:55.454378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec3a9e-83e7-4c25-bec1-090ef0f82212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:55.455837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec3a9e-83e7-4c25-bec1-090ef0f82212 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-a6ec3a9e-83e7-4c25-bec1-090ef0f82212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:55.472309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:55.488742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:55.489704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:55.489704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:55.489704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:55.489704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 413/837] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:55.489947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976a9b2-85a1-4ea0-a449-bedb2132ecab None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:55.589221 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976a9b2-85a1-4ea0-a449-bedb2132ecab None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:55.601478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:55.625871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686b953-89be-40fb-82ff-31cde74f5431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:55.626701 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686b953-89be-40fb-82ff-31cde74f5431 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:55.627086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686b953-89be-40fb-82ff-31cde74f5431 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:55.627502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686b953-89be-40fb-82ff-31cde74f5431 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:55.628270 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686b953-89be-40fb-82ff-31cde74f5431 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:55.628680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686b953-89be-40fb-82ff-31cde74f5431 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:55.630162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686b953-89be-40fb-82ff-31cde74f5431 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:55.646686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 425/838] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:55.649260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec3a9e-83e7-4c25-bec1-090ef0f82212 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:55.761665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec3a9e-83e7-4c25-bec1-090ef0f82212 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:55.799076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83766-81fd-4c0b-86ee-a9287349c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:55.799957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83766-81fd-4c0b-86ee-a9287349c5e4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:55.800391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83766-81fd-4c0b-86ee-a9287349c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:55.800813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83766-81fd-4c0b-86ee-a9287349c5e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:55.809228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83766-81fd-4c0b-86ee-a9287349c5e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:55.810881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83766-81fd-4c0b-86ee-a9287349c5e4 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-a0a83766-81fd-4c0b-86ee-a9287349c5e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:55.823508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:55.824706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:55.825926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:55.825926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:55.825926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:55.825926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:55.825926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:55.825926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:55.825926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:55.825926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:55.825926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 18:33:55.825926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:55.828108 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 414/839] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:33:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:55.833556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686b953-89be-40fb-82ff-31cde74f5431 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:55.940372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686b953-89be-40fb-82ff-31cde74f5431 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:55.977374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8e4a9-1598-4aa4-8311-ef8bdcf55f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:55.978418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8e4a9-1598-4aa4-8311-ef8bdcf55f47 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:55.978702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8e4a9-1598-4aa4-8311-ef8bdcf55f47 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:55.979022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8e4a9-1598-4aa4-8311-ef8bdcf55f47 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:55.979762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8e4a9-1598-4aa4-8311-ef8bdcf55f47 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:55.980074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8e4a9-1598-4aa4-8311-ef8bdcf55f47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:55.981544 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8e4a9-1598-4aa4-8311-ef8bdcf55f47 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:56.000949 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 426/840] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 8 18:33:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:56.006670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83766-81fd-4c0b-86ee-a9287349c5e4 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:56.064711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:56.118832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83766-81fd-4c0b-86ee-a9287349c5e4 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:56.158277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6836cd-402a-45f6-94f1-b581d6335892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:56.159466 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6836cd-402a-45f6-94f1-b581d6335892 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:56.159980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6836cd-402a-45f6-94f1-b581d6335892 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:56.160542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6836cd-402a-45f6-94f1-b581d6335892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:56.240206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6836cd-402a-45f6-94f1-b581d6335892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:56.242352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6836cd-402a-45f6-94f1-b581d6335892 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:56.277323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056075f8-748c-44fe-b526-a4c53eb35b90 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:56.278217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056075f8-748c-44fe-b526-a4c53eb35b90 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:56.278736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056075f8-748c-44fe-b526-a4c53eb35b90 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:56.285278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 427/841] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:33:56.286286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfaa55-68a8-46e4-b69e-5bf4456ebc41 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:56.286568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfaa55-68a8-46e4-b69e-5bf4456ebc41 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:56.286842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfaa55-68a8-46e4-b69e-5bf4456ebc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:56.306505 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-afdfaa55-68a8-46e4-b69e-5bf4456ebc41 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:56.307695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-afdfaa55-68a8-46e4-b69e-5bf4456ebc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:56.308901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:56.344624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:56.373264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 415/842] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:33:56.374225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6836cd-402a-45f6-94f1-b581d6335892 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:56.411232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 428/843] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:33:56.491181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6836cd-402a-45f6-94f1-b581d6335892 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:56.529412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ce705-2259-44e5-ac71-48f656dbaae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:56.530391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ce705-2259-44e5-ac71-48f656dbaae6 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:56.530873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ce705-2259-44e5-ac71-48f656dbaae6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:56.531347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ce705-2259-44e5-ac71-48f656dbaae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:56.542178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ce705-2259-44e5-ac71-48f656dbaae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:56.543835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ce705-2259-44e5-ac71-48f656dbaae6 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-a42ce705-2259-44e5-ac71-48f656dbaae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:56.560229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:56.561300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:56.562212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:56.562212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:56.562212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:56.564311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 416/844] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:56.574180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfaa55-68a8-46e4-b69e-5bf4456ebc41 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:56.693219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfaa55-68a8-46e4-b69e-5bf4456ebc41 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:56.720533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ce705-2259-44e5-ac71-48f656dbaae6 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:56.724451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:56.736790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc39dca-89de-479b-a6c6-8dfdcb6d5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:56.737469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc39dca-89de-479b-a6c6-8dfdcb6d5e71 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:56.737752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc39dca-89de-479b-a6c6-8dfdcb6d5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:56.738025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc39dca-89de-479b-a6c6-8dfdcb6d5e71 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:56.741837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc39dca-89de-479b-a6c6-8dfdcb6d5e71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:56.741837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc39dca-89de-479b-a6c6-8dfdcb6d5e71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:56.742037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc39dca-89de-479b-a6c6-8dfdcb6d5e71 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:56.764455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 429/845] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:56.766522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc39dca-89de-479b-a6c6-8dfdcb6d5e71 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:56.790850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:56.849491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ce705-2259-44e5-ac71-48f656dbaae6 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:56.885494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2b0dd-e80a-4e38-9561-d2efca32c46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:56.886126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2b0dd-e80a-4e38-9561-d2efca32c46c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:56.886407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2b0dd-e80a-4e38-9561-d2efca32c46c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:56.886685 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2b0dd-e80a-4e38-9561-d2efca32c46c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:56.887269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2b0dd-e80a-4e38-9561-d2efca32c46c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:56.888632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2b0dd-e80a-4e38-9561-d2efca32c46c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:56.891519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc39dca-89de-479b-a6c6-8dfdcb6d5e71 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-e5b2b0dd-e80a-4e38-9561-d2efca32c46c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:56.898441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:56.902819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:56.903794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 417/846] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:33:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:56.904595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3963d-5cf5-4f0a-884b-31da2fc2435e None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:56.904595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3963d-5cf5-4f0a-884b-31da2fc2435e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:56.904595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3963d-5cf5-4f0a-884b-31da2fc2435e None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:56.904595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 418/847] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:56.905382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3963d-5cf5-4f0a-884b-31da2fc2435e None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:56.936183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb5cf0-1320-4df2-9a25-0798dc46ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:56.936852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb5cf0-1320-4df2-9a25-0798dc46ded1 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:56.937285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb5cf0-1320-4df2-9a25-0798dc46ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:56.937589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb5cf0-1320-4df2-9a25-0798dc46ded1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:56.945185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb5cf0-1320-4df2-9a25-0798dc46ded1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:56.946615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb5cf0-1320-4df2-9a25-0798dc46ded1 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-beeb5cf0-1320-4df2-9a25-0798dc46ded1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:33:56.957896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:56.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:56.959863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:33:56.959863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:33:56.959863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:33:56.959863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:33:56.959863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:33:56.959863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:33:56.959863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:33:56.959863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:56.959863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 430/848] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:56.960413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340374d-1c1f-4572-a200-eaec37140c19 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:56.960544 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340374d-1c1f-4572-a200-eaec37140c19 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:56.961136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340374d-1c1f-4572-a200-eaec37140c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:56.979381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-0340374d-1c1f-4572-a200-eaec37140c19 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:56.980580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0340374d-1c1f-4572-a200-eaec37140c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:56.989134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:57.028163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3963d-5cf5-4f0a-884b-31da2fc2435e None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:57.056342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 431/849] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:33:57.057673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340374d-1c1f-4572-a200-eaec37140c19 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:57.071298 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333bfa9-5fbf-4f7f-9d27-1801a7f1dcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:57.072389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333bfa9-5fbf-4f7f-9d27-1801a7f1dcae None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:57.072885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333bfa9-5fbf-4f7f-9d27-1801a7f1dcae None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:57.073409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333bfa9-5fbf-4f7f-9d27-1801a7f1dcae None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:57.074396 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:57.075149 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333bfa9-5fbf-4f7f-9d27-1801a7f1dcae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:57.075805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333bfa9-5fbf-4f7f-9d27-1801a7f1dcae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:57.077453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333bfa9-5fbf-4f7f-9d27-1801a7f1dcae None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:57.082898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 419/850] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:33:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:57.088609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333bfa9-5fbf-4f7f-9d27-1801a7f1dcae None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:57.196983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340374d-1c1f-4572-a200-eaec37140c19 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:57.224623 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333bfa9-5fbf-4f7f-9d27-1801a7f1dcae None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:57.240522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3482c-a9d8-4ba4-b030-418693eeb0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:57.241318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3482c-a9d8-4ba4-b030-418693eeb0a8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:57.241605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3482c-a9d8-4ba4-b030-418693eeb0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:57.241882 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3482c-a9d8-4ba4-b030-418693eeb0a8 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:57.242583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3482c-a9d8-4ba4-b030-418693eeb0a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:57.242893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3482c-a9d8-4ba4-b030-418693eeb0a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:57.244882 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3482c-a9d8-4ba4-b030-418693eeb0a8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:57.260543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 432/851] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:33:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:57.265645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3482c-a9d8-4ba4-b030-418693eeb0a8 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:57.271062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:57.277515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066e075-3a35-4e4e-be65-dea7493aff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:57.278605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066e075-3a35-4e4e-be65-dea7493aff6d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:57.278865 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066e075-3a35-4e4e-be65-dea7493aff6d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:57.279270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066e075-3a35-4e4e-be65-dea7493aff6d None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:57.279916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066e075-3a35-4e4e-be65-dea7493aff6d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:57.280225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066e075-3a35-4e4e-be65-dea7493aff6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:57.282391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066e075-3a35-4e4e-be65-dea7493aff6d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:57.298007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 420/852] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:33:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:57.336782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066e075-3a35-4e4e-be65-dea7493aff6d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:57.369693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:57.417893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3482c-a9d8-4ba4-b030-418693eeb0a8 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:57.462125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066e075-3a35-4e4e-be65-dea7493aff6d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:57.463444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3f72d-13e7-4fb7-bbb1-5bc26b831994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:57.464072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3f72d-13e7-4fb7-bbb1-5bc26b831994 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:57.464350 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3f72d-13e7-4fb7-bbb1-5bc26b831994 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:57.464719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3f72d-13e7-4fb7-bbb1-5bc26b831994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:57.465629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3f72d-13e7-4fb7-bbb1-5bc26b831994 None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=88a9b2cee21f4831b221d113ddbbd29d) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:57.465941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3f72d-13e7-4fb7-bbb1-5bc26b831994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:57.467529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3f72d-13e7-4fb7-bbb1-5bc26b831994 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:57.475644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 433/853] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:33:57 2022] GET /identity/v3/role_assignments?user.id=88a9b2cee21f4831b221d113ddbbd29d&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=5c777de40b8449329d714d8c2229115e => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:57.506298 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225017c-a11d-4afc-a7d7-f5a7d9cf016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:57.507077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225017c-a11d-4afc-a7d7-f5a7d9cf016d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:57.507430 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225017c-a11d-4afc-a7d7-f5a7d9cf016d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:57.507781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225017c-a11d-4afc-a7d7-f5a7d9cf016d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:57.508476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225017c-a11d-4afc-a7d7-f5a7d9cf016d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:57.510000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225017c-a11d-4afc-a7d7-f5a7d9cf016d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-7225017c-a11d-4afc-a7d7-f5a7d9cf016d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:57.521574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:57.523238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:57.524252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 421/854] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 8 18:33:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:33:57.529680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3f72d-13e7-4fb7-bbb1-5bc26b831994 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:57.553866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb11f4-40e1-4fc9-aec0-c7f2f373082b None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:57.554407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb11f4-40e1-4fc9-aec0-c7f2f373082b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:57.554881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb11f4-40e1-4fc9-aec0-c7f2f373082b None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:57.556778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 422/855] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:33:57.562263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc527794-7b75-4b79-b1a6-07f137ce544e None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:57.562779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc527794-7b75-4b79-b1a6-07f137ce544e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:57.563232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc527794-7b75-4b79-b1a6-07f137ce544e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:57.583984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-cc527794-7b75-4b79-b1a6-07f137ce544e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:57.586270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cc527794-7b75-4b79-b1a6-07f137ce544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:57.659372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3f72d-13e7-4fb7-bbb1-5bc26b831994 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:57.683375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:57.700071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 423/856] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:33:57.708911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:57.717894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbc1fd-9b08-4a9a-ae6b-4572a3d5c7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:57.717894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbc1fd-9b08-4a9a-ae6b-4572a3d5c7ed None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:57.717894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbc1fd-9b08-4a9a-ae6b-4572a3d5c7ed None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:57.719291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbc1fd-9b08-4a9a-ae6b-4572a3d5c7ed None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:57.719291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbc1fd-9b08-4a9a-ae6b-4572a3d5c7ed None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:57.721079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbc1fd-9b08-4a9a-ae6b-4572a3d5c7ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:57.721228 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbc1fd-9b08-4a9a-ae6b-4572a3d5c7ed None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:57.726901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 434/857] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:33:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:57.733290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc527794-7b75-4b79-b1a6-07f137ce544e None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:57.885388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc527794-7b75-4b79-b1a6-07f137ce544e None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:57.931777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a558eb-7edc-40ba-8eb4-b35c393ecc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:57.936557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a558eb-7edc-40ba-8eb4-b35c393ecc7c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:57.936557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a558eb-7edc-40ba-8eb4-b35c393ecc7c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:57.936557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a558eb-7edc-40ba-8eb4-b35c393ecc7c None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:57.936557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a558eb-7edc-40ba-8eb4-b35c393ecc7c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:57.936557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a558eb-7edc-40ba-8eb4-b35c393ecc7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:57.936557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a558eb-7edc-40ba-8eb4-b35c393ecc7c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:57.950316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 424/858] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:33:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:33:58.072747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbc1fd-9b08-4a9a-ae6b-4572a3d5c7ed None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:58.206992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbc1fd-9b08-4a9a-ae6b-4572a3d5c7ed None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:58.244318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675be8a-d7ba-419c-827e-c18d0e2dc8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:58.245275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675be8a-d7ba-419c-827e-c18d0e2dc8d1 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:58.245706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675be8a-d7ba-419c-827e-c18d0e2dc8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:58.246127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675be8a-d7ba-419c-827e-c18d0e2dc8d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:58.257321 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675be8a-d7ba-419c-827e-c18d0e2dc8d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-c675be8a-d7ba-419c-827e-c18d0e2dc8d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:33:58.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:58.264028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:33:58.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:58.267148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 435/859] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:33:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:58.278952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a558eb-7edc-40ba-8eb4-b35c393ecc7c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:58.358023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:58.405301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a558eb-7edc-40ba-8eb4-b35c393ecc7c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:58.443074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9bb50-b79c-49d2-a17c-0189b797ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:58.443703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9bb50-b79c-49d2-a17c-0189b797ffe0 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:58.444713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9bb50-b79c-49d2-a17c-0189b797ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:58.445138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9bb50-b79c-49d2-a17c-0189b797ffe0 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:58.445770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9bb50-b79c-49d2-a17c-0189b797ffe0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:58.446073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9bb50-b79c-49d2-a17c-0189b797ffe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:58.447516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9bb50-b79c-49d2-a17c-0189b797ffe0 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:58.456629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 425/860] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:33:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:58.462696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675be8a-d7ba-419c-827e-c18d0e2dc8d1 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:58.526806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:58.575302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675be8a-d7ba-419c-827e-c18d0e2dc8d1 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:58.612012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53ed73-589c-4ca9-9fd1-e73ea1c7a520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:58.612654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53ed73-589c-4ca9-9fd1-e73ea1c7a520 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:58.612938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53ed73-589c-4ca9-9fd1-e73ea1c7a520 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:58.613236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53ed73-589c-4ca9-9fd1-e73ea1c7a520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:58.624812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53ed73-589c-4ca9-9fd1-e73ea1c7a520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:58.626430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53ed73-589c-4ca9-9fd1-e73ea1c7a520 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-5f53ed73-589c-4ca9-9fd1-e73ea1c7a520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:58.641618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:58.642589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:58.643541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:58.643541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:58.643541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:58.645639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 436/861] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:58.651523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9bb50-b79c-49d2-a17c-0189b797ffe0 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:58.757323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9bb50-b79c-49d2-a17c-0189b797ffe0 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:58.774170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:58.795277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa75297-8894-4963-9578-e01f3ae59e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:58.796282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa75297-8894-4963-9578-e01f3ae59e64 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:58.796772 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa75297-8894-4963-9578-e01f3ae59e64 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:58.797286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa75297-8894-4963-9578-e01f3ae59e64 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:58.798442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa75297-8894-4963-9578-e01f3ae59e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:58.798998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa75297-8894-4963-9578-e01f3ae59e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:58.801595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa75297-8894-4963-9578-e01f3ae59e64 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:58.816408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 426/862] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:58.824250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53ed73-589c-4ca9-9fd1-e73ea1c7a520 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:58.935313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53ed73-589c-4ca9-9fd1-e73ea1c7a520 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:58.944584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:58.970221 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1686a41-c63d-4e36-b800-8f4b38f5baee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:58.970856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1686a41-c63d-4e36-b800-8f4b38f5baee None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:58.971197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1686a41-c63d-4e36-b800-8f4b38f5baee None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:58.971433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1686a41-c63d-4e36-b800-8f4b38f5baee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:58.979683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1686a41-c63d-4e36-b800-8f4b38f5baee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:58.981134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1686a41-c63d-4e36-b800-8f4b38f5baee None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-b1686a41-c63d-4e36-b800-8f4b38f5baee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:33:58.992893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:58.993872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:33:58.994842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 437/863] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 18:33:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:59.001401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa75297-8894-4963-9578-e01f3ae59e64 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:59.111626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa75297-8894-4963-9578-e01f3ae59e64 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:59.148979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5597f6-61c8-46b4-8021-25509aad8aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:59.149968 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5597f6-61c8-46b4-8021-25509aad8aa8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:59.150456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5597f6-61c8-46b4-8021-25509aad8aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:59.150937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5597f6-61c8-46b4-8021-25509aad8aa8 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:59.151860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5597f6-61c8-46b4-8021-25509aad8aa8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:59.152446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5597f6-61c8-46b4-8021-25509aad8aa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:59.154110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5597f6-61c8-46b4-8021-25509aad8aa8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:59.172820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 427/864] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 18:33:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:59.179281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1686a41-c63d-4e36-b800-8f4b38f5baee None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:59.283585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1686a41-c63d-4e36-b800-8f4b38f5baee None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:59.318371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4c8f3-ced1-4a74-921e-d7cb0bb2d1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:59.319517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4c8f3-ced1-4a74-921e-d7cb0bb2d1eb None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:59.319794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4c8f3-ced1-4a74-921e-d7cb0bb2d1eb None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:59.320106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4c8f3-ced1-4a74-921e-d7cb0bb2d1eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:59.356790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:59.392358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4c8f3-ced1-4a74-921e-d7cb0bb2d1eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:59.393758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4c8f3-ced1-4a74-921e-d7cb0bb2d1eb None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:59.496318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 438/865] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:33:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:33:59.502044 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5597f6-61c8-46b4-8021-25509aad8aa8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:59.540431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:59.613920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5597f6-61c8-46b4-8021-25509aad8aa8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:59.652020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01677521-fc5f-4b1b-bd8a-03ed9894c00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:59.652694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01677521-fc5f-4b1b-bd8a-03ed9894c00e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:59.652982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01677521-fc5f-4b1b-bd8a-03ed9894c00e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:59.653280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01677521-fc5f-4b1b-bd8a-03ed9894c00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:59.663895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01677521-fc5f-4b1b-bd8a-03ed9894c00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:59.665393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01677521-fc5f-4b1b-bd8a-03ed9894c00e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-01677521-fc5f-4b1b-bd8a-03ed9894c00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:33:59.683018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:33:59.684052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:33:59.684949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:33:59.684949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:33:59.684949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:33:59.684949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 428/866] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:33:59.691776 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4c8f3-ced1-4a74-921e-d7cb0bb2d1eb None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:59.768918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:59.804137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45666e8-029e-47e8-8711-6e7348f76ddd None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:59.804335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45666e8-029e-47e8-8711-6e7348f76ddd None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:59.804801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45666e8-029e-47e8-8711-6e7348f76ddd None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:59.806423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4c8f3-ced1-4a74-921e-d7cb0bb2d1eb None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:33:59.806542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 429/867] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:59.811580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c6e71-9aa2-4f06-9c65-4dfc91b97df6 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:59.811861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c6e71-9aa2-4f06-9c65-4dfc91b97df6 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:59.812135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c6e71-9aa2-4f06-9c65-4dfc91b97df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:59.830469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-697c6e71-9aa2-4f06-9c65-4dfc91b97df6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:59.831658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-697c6e71-9aa2-4f06-9c65-4dfc91b97df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:59.842236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04787a6b-c2fd-4219-925b-4c45cec9f439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:33:59.842864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04787a6b-c2fd-4219-925b-4c45cec9f439 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:59.843125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04787a6b-c2fd-4219-925b-4c45cec9f439 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:59.843660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04787a6b-c2fd-4219-925b-4c45cec9f439 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:59.844367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04787a6b-c2fd-4219-925b-4c45cec9f439 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:33:59.844674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04787a6b-c2fd-4219-925b-4c45cec9f439 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:33:59.846060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04787a6b-c2fd-4219-925b-4c45cec9f439 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:33:59.859111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 439/868] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:33:59.864514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04787a6b-c2fd-4219-925b-4c45cec9f439 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:33:59.925182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 430/869] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:33:59.925182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296a4b-03fd-4256-b4d9-c16714cc9a1c None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:59.925182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296a4b-03fd-4256-b4d9-c16714cc9a1c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:59.925182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc296a4b-03fd-4256-b4d9-c16714cc9a1c None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:59.925182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 431/870] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:33:59.929659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3b26-b7fe-4820-9cbb-d5b0f791b3e4 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:33:59.929943 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3b26-b7fe-4820-9cbb-d5b0f791b3e4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:33:59.930491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c3b26-b7fe-4820-9cbb-d5b0f791b3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:33:59.951060 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ef6c3b26-b7fe-4820-9cbb-d5b0f791b3e4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:33:59.952775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ef6c3b26-b7fe-4820-9cbb-d5b0f791b3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:33:59.967236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:33:59.982981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04787a6b-c2fd-4219-925b-4c45cec9f439 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:00.013940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:00.024981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0fd87-99f7-4b65-8031-c9f6dc93bda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:00.025645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0fd87-99f7-4b65-8031-c9f6dc93bda4 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:00.025923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0fd87-99f7-4b65-8031-c9f6dc93bda4 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:00.026194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0fd87-99f7-4b65-8031-c9f6dc93bda4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:00.035447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0fd87-99f7-4b65-8031-c9f6dc93bda4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:00.036931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0fd87-99f7-4b65-8031-c9f6dc93bda4 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:00.045133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 432/871] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:33:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-faa0fd87-99f7-4b65-8031-c9f6dc93bda4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:34:00.056934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:00.057854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:00.059027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:34:00.059027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:34:00.059027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:34:00.059027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:34:00.059027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:34:00.059027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:34:00.059027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:00.059027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:00.059027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 440/872] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:33:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:00.064064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c3b26-b7fe-4820-9cbb-d5b0f791b3e4 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:00.169410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c3b26-b7fe-4820-9cbb-d5b0f791b3e4 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:00.207541 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d486b0-86ba-491d-abcf-51fe7e71e65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:00.207724 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0fd87-99f7-4b65-8031-c9f6dc93bda4 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:00.208072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d486b0-86ba-491d-abcf-51fe7e71e65e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:00.208347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d486b0-86ba-491d-abcf-51fe7e71e65e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:00.208617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d486b0-86ba-491d-abcf-51fe7e71e65e None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:00.209446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d486b0-86ba-491d-abcf-51fe7e71e65e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:00.209727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d486b0-86ba-491d-abcf-51fe7e71e65e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:00.211155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d486b0-86ba-491d-abcf-51fe7e71e65e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:00.229221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 433/873] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:34:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:00.235376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d486b0-86ba-491d-abcf-51fe7e71e65e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:00.245537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:00.326359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0fd87-99f7-4b65-8031-c9f6dc93bda4 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:00.341388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d486b0-86ba-491d-abcf-51fe7e71e65e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:00.362629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da825af-61fd-417b-9dab-df2532b8192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:00.363244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da825af-61fd-417b-9dab-df2532b8192f None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:00.363525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da825af-61fd-417b-9dab-df2532b8192f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:00.363799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da825af-61fd-417b-9dab-df2532b8192f None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:00.364476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da825af-61fd-417b-9dab-df2532b8192f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:00.366426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da825af-61fd-417b-9dab-df2532b8192f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:00.375986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b231ff3-6ab6-4e14-9590-cfa5875059e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:00.376583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b231ff3-6ab6-4e14-9590-cfa5875059e5 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:00.376861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b231ff3-6ab6-4e14-9590-cfa5875059e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:00.377166 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b231ff3-6ab6-4e14-9590-cfa5875059e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:00.377883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b231ff3-6ab6-4e14-9590-cfa5875059e5 None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=3038d1d505654ffb95b78fbc9005baf4) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:00.378190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b231ff3-6ab6-4e14-9590-cfa5875059e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:00.379596 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b231ff3-6ab6-4e14-9590-cfa5875059e5 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:00.386202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 434/874] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:34:00 2022] GET /identity/v3/role_assignments?user.id=3038d1d505654ffb95b78fbc9005baf4&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:00.387512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51092a25-16bb-419d-b21a-5d0ac32f0121 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:00.387800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51092a25-16bb-419d-b21a-5d0ac32f0121 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:00.388068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51092a25-16bb-419d-b21a-5d0ac32f0121 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:00.389305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 435/875] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:00.391429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51092a25-16bb-419d-b21a-5d0ac32f0121 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:00.442171 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 441/876] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:34:00 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:34:00.444435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45419097-4bcc-4dfc-82e6-1a1c31a6ab2e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:00.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45419097-4bcc-4dfc-82e6-1a1c31a6ab2e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:00.445844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45419097-4bcc-4dfc-82e6-1a1c31a6ab2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:00.473392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-45419097-4bcc-4dfc-82e6-1a1c31a6ab2e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:00.474609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-45419097-4bcc-4dfc-82e6-1a1c31a6ab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:00.503194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:00.533816 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:00.579747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 442/877] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:00 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:00.580018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51092a25-16bb-419d-b21a-5d0ac32f0121 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:00.630651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8925221-fed8-43d6-9aed-0a52fab5e6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:00.632474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8925221-fed8-43d6-9aed-0a52fab5e6d8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:00.633141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8925221-fed8-43d6-9aed-0a52fab5e6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:00.633569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8925221-fed8-43d6-9aed-0a52fab5e6d8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:00.634304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8925221-fed8-43d6-9aed-0a52fab5e6d8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:00.635849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8925221-fed8-43d6-9aed-0a52fab5e6d8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-a8925221-fed8-43d6-9aed-0a52fab5e6d8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:00.645386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:00.646605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:00.649001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 436/878] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:34:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:00.655920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45419097-4bcc-4dfc-82e6-1a1c31a6ab2e None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:00.766087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45419097-4bcc-4dfc-82e6-1a1c31a6ab2e None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:00.809551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09876ac-e04d-4281-83a1-de3951c26ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:00.810170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09876ac-e04d-4281-83a1-de3951c26ca9 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:00.810448 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09876ac-e04d-4281-83a1-de3951c26ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:00.810718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09876ac-e04d-4281-83a1-de3951c26ca9 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:00.811342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09876ac-e04d-4281-83a1-de3951c26ca9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:00.811642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09876ac-e04d-4281-83a1-de3951c26ca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:00.812995 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09876ac-e04d-4281-83a1-de3951c26ca9 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:00.818036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 443/879] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 18:34:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:00.828620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8925221-fed8-43d6-9aed-0a52fab5e6d8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:00.882765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09876ac-e04d-4281-83a1-de3951c26ca9 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:00.940044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:00.954037 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8925221-fed8-43d6-9aed-0a52fab5e6d8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:00.995115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:01.002405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b64926-7f06-4d9f-94d1-9981e69386ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:01.003020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b64926-7f06-4d9f-94d1-9981e69386ad None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:01.003302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b64926-7f06-4d9f-94d1-9981e69386ad None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:01.003715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b64926-7f06-4d9f-94d1-9981e69386ad None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:01.004501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b64926-7f06-4d9f-94d1-9981e69386ad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:01.004811 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b64926-7f06-4d9f-94d1-9981e69386ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:01.008020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b64926-7f06-4d9f-94d1-9981e69386ad None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:01.014487 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 437/880] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 18:34:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:01.017126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09876ac-e04d-4281-83a1-de3951c26ca9 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:01.063201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593ace8-49eb-485f-9e53-4c531ba1a383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:01.064210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ace8-49eb-485f-9e53-4c531ba1a383 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:01.064696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ace8-49eb-485f-9e53-4c531ba1a383 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:01.065209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ace8-49eb-485f-9e53-4c531ba1a383 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:01.066101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593ace8-49eb-485f-9e53-4c531ba1a383 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:01.067735 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593ace8-49eb-485f-9e53-4c531ba1a383 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:01.179478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 444/881] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:34:00 2022] POST /identity/v3/services => generated 269 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:34:02.703453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131f378-04dd-443e-a862-22bdabe0c6e0 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:02.703964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131f378-04dd-443e-a862-22bdabe0c6e0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:02.704040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131f378-04dd-443e-a862-22bdabe0c6e0 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:02.705618 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 438/882] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:02.711636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc26f-55ba-4830-ae77-3e3fd4d36a63 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:02.712552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc26f-55ba-4830-ae77-3e3fd4d36a63 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:02.713322 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc26f-55ba-4830-ae77-3e3fd4d36a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:02.735117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b44dc26f-55ba-4830-ae77-3e3fd4d36a63 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:02.736582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b44dc26f-55ba-4830-ae77-3e3fd4d36a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:02.829806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 445/883] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:02.869802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f3605-d60a-43c0-944e-a623ec565499 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:02.870048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f3605-d60a-43c0-944e-a623ec565499 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:02.870433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f3605-d60a-43c0-944e-a623ec565499 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:02.872079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 439/884] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:02.877326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6542b1-d2ad-4e14-a472-301e11e7aa22 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:02.877616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6542b1-d2ad-4e14-a472-301e11e7aa22 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:02.877890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6542b1-d2ad-4e14-a472-301e11e7aa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:02.896388 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-af6542b1-d2ad-4e14-a472-301e11e7aa22 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:02.898169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-af6542b1-d2ad-4e14-a472-301e11e7aa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:02.917697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:02.971014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 446/885] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:03.117935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f3605-d60a-43c0-944e-a623ec565499 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:03.234131 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f3605-d60a-43c0-944e-a623ec565499 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:03.258937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6542b1-d2ad-4e14-a472-301e11e7aa22 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:03.281852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeffffa-9bf2-4e73-a96f-d161c4a6df0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:03.282515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeffffa-9bf2-4e73-a96f-d161c4a6df0a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.282796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeffffa-9bf2-4e73-a96f-d161c4a6df0a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.283073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeffffa-9bf2-4e73-a96f-d161c4a6df0a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.283683 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeffffa-9bf2-4e73-a96f-d161c4a6df0a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:03.285467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeffffa-9bf2-4e73-a96f-d161c4a6df0a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-5aeffffa-9bf2-4e73-a96f-d161c4a6df0a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:03.300681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:03.301858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 18:34:03.302761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:03.306114 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 440/886] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:34:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:03.321072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeffffa-9bf2-4e73-a96f-d161c4a6df0a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:03.377513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:03.379308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6542b1-d2ad-4e14-a472-301e11e7aa22 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:03.415801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccad8d-7fac-4382-9022-a18ee3a39d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:03.416445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccad8d-7fac-4382-9022-a18ee3a39d91 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.416725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccad8d-7fac-4382-9022-a18ee3a39d91 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.416997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccad8d-7fac-4382-9022-a18ee3a39d91 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.421319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeffffa-9bf2-4e73-a96f-d161c4a6df0a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:03.427348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccad8d-7fac-4382-9022-a18ee3a39d91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-edccad8d-7fac-4382-9022-a18ee3a39d91 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:34:03.432914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:03.433976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:03.434846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 447/887] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 18:34:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:03.435573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066e814-66bc-427d-a288-0520ca5964ba None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.435812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066e814-66bc-427d-a288-0520ca5964ba None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.436132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066e814-66bc-427d-a288-0520ca5964ba None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.441907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 448/888] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:03.441907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017cdaa-0fa9-484f-be6c-72935a2af9bd None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.441907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017cdaa-0fa9-484f-be6c-72935a2af9bd None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.441907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017cdaa-0fa9-484f-be6c-72935a2af9bd None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.441907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 449/889] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:03.441907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017cdaa-0fa9-484f-be6c-72935a2af9bd None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:03.457513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:03.462983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51100bd-df3d-4836-82ee-1bf2915456f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:03.463650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51100bd-df3d-4836-82ee-1bf2915456f7 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.463876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51100bd-df3d-4836-82ee-1bf2915456f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.464158 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51100bd-df3d-4836-82ee-1bf2915456f7 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.464779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51100bd-df3d-4836-82ee-1bf2915456f7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:03.465115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51100bd-df3d-4836-82ee-1bf2915456f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:03.469395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51100bd-df3d-4836-82ee-1bf2915456f7 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:03.471620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 441/890] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:34:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:03.472504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7d80d-fda3-4fc7-ab9c-2afcd9455f9b None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.472844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7d80d-fda3-4fc7-ab9c-2afcd9455f9b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.473078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7d80d-fda3-4fc7-ab9c-2afcd9455f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.492007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-7cc7d80d-fda3-4fc7-ab9c-2afcd9455f9b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:03.493161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7cc7d80d-fda3-4fc7-ab9c-2afcd9455f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:03.545088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017cdaa-0fa9-484f-be6c-72935a2af9bd None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:03.561035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 442/891] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:03.562294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824624ab-6021-4a02-99a4-3d323d62aff1 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.562584 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824624ab-6021-4a02-99a4-3d323d62aff1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.562865 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824624ab-6021-4a02-99a4-3d323d62aff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.580831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20a3f4-6ab7-45ec-9db9-f2a47c2a631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:03.581484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20a3f4-6ab7-45ec-9db9-f2a47c2a631b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.581766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20a3f4-6ab7-45ec-9db9-f2a47c2a631b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.582042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20a3f4-6ab7-45ec-9db9-f2a47c2a631b None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.582704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20a3f4-6ab7-45ec-9db9-f2a47c2a631b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:03.583011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20a3f4-6ab7-45ec-9db9-f2a47c2a631b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:03.584870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20a3f4-6ab7-45ec-9db9-f2a47c2a631b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:03.589518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-824624ab-6021-4a02-99a4-3d323d62aff1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:03.590926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-824624ab-6021-4a02-99a4-3d323d62aff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:03.602811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 450/892] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 18:34:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:03.602811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20a3f4-6ab7-45ec-9db9-f2a47c2a631b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:03.645182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:03.666644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 443/893] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:03.667805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824624ab-6021-4a02-99a4-3d323d62aff1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:03.706378 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:03.733222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20a3f4-6ab7-45ec-9db9-f2a47c2a631b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:03.772452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1e7d-6066-42b7-b268-b26f2c2fb3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:03.773341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1e7d-6066-42b7-b268-b26f2c2fb3e7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.773825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1e7d-6066-42b7-b268-b26f2c2fb3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.774279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1e7d-6066-42b7-b268-b26f2c2fb3e7 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.775199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def1e7d-6066-42b7-b268-b26f2c2fb3e7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:03.775750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def1e7d-6066-42b7-b268-b26f2c2fb3e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:03.777383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def1e7d-6066-42b7-b268-b26f2c2fb3e7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:03.781562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824624ab-6021-4a02-99a4-3d323d62aff1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:03.789400 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 451/894] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:34:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:03.790888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1e7d-6066-42b7-b268-b26f2c2fb3e7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:03.819247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7a4b-5e13-454a-a01a-59c610a7b23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:03.819708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7a4b-5e13-454a-a01a-59c610a7b23a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.820066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7a4b-5e13-454a-a01a-59c610a7b23a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.820710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7a4b-5e13-454a-a01a-59c610a7b23a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.831768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f7a4b-5e13-454a-a01a-59c610a7b23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:03.833072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f7a4b-5e13-454a-a01a-59c610a7b23a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-144f7a4b-5e13-454a-a01a-59c610a7b23a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:03.848628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:03.849692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:03.856388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:03.856388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:03.856388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:03.856388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 444/895] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:03.856388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7a4b-5e13-454a-a01a-59c610a7b23a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:03.887689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:03.905004 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1e7d-6066-42b7-b268-b26f2c2fb3e7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:03.940614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c9cb9-348c-4985-a911-8166627bf31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:03.941640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c9cb9-348c-4985-a911-8166627bf31f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.941944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c9cb9-348c-4985-a911-8166627bf31f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.942198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c9cb9-348c-4985-a911-8166627bf31f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.943071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c9cb9-348c-4985-a911-8166627bf31f None admin] RBAC: Adding query filter params (interface=public, service_id=74c44b4cbac7423a90db4f9d4bbbc44f, region_id=RegionOne) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:03.943381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c9cb9-348c-4985-a911-8166627bf31f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:03.945295 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c9cb9-348c-4985-a911-8166627bf31f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:03.952638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 452/896] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 18:34:03 2022] GET /identity/v3/endpoints?service_id=74c44b4cbac7423a90db4f9d4bbbc44f&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:03.953471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c9cb9-348c-4985-a911-8166627bf31f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:03.957637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:03.962231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7a4b-5e13-454a-a01a-59c610a7b23a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:03.998183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16fb6f-d31b-449d-8f1c-e758bfda3f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:03.998784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16fb6f-d31b-449d-8f1c-e758bfda3f1b None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:03.999062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16fb6f-d31b-449d-8f1c-e758bfda3f1b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:03.999332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16fb6f-d31b-449d-8f1c-e758bfda3f1b None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:03.999970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16fb6f-d31b-449d-8f1c-e758bfda3f1b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:04.001361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b16fb6f-d31b-449d-8f1c-e758bfda3f1b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:04.053945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c9cb9-348c-4985-a911-8166627bf31f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:04.071631 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 445/897] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:34:03 2022] POST /identity/v3/services => generated 241 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:34:04.075848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16fb6f-d31b-449d-8f1c-e758bfda3f1b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:04.099692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c32ff-621c-4e85-8cd6-3e64d51bc84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:04.101220 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c32ff-621c-4e85-8cd6-3e64d51bc84c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:04.101529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c32ff-621c-4e85-8cd6-3e64d51bc84c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:04.101802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c32ff-621c-4e85-8cd6-3e64d51bc84c None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:04.102483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c32ff-621c-4e85-8cd6-3e64d51bc84c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:04.102789 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c32ff-621c-4e85-8cd6-3e64d51bc84c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:04.104267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c32ff-621c-4e85-8cd6-3e64d51bc84c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:04.118197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 453/898] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:04.118902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c32ff-621c-4e85-8cd6-3e64d51bc84c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:04.189537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:04.200838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:04.208507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16fb6f-d31b-449d-8f1c-e758bfda3f1b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:04.237345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c32ff-621c-4e85-8cd6-3e64d51bc84c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:04.253181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ab733-b8cd-486e-9034-8155294f2baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:04.253817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ab733-b8cd-486e-9034-8155294f2baa None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:04.254523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ab733-b8cd-486e-9034-8155294f2baa None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:04.254799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ab733-b8cd-486e-9034-8155294f2baa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:04.255390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ab733-b8cd-486e-9034-8155294f2baa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:04.256846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ab733-b8cd-486e-9034-8155294f2baa None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-c80ab733-b8cd-486e-9034-8155294f2baa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:04.267072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:04.268089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:04.268987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 446/899] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 8 18:34:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:04.269916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ab733-b8cd-486e-9034-8155294f2baa None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:04.281072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c18f71-3d79-44ea-879e-7f8a6ffded8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:04.281863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c18f71-3d79-44ea-879e-7f8a6ffded8d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:04.282207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c18f71-3d79-44ea-879e-7f8a6ffded8d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:04.282852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c18f71-3d79-44ea-879e-7f8a6ffded8d None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:04.283804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c18f71-3d79-44ea-879e-7f8a6ffded8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:04.286461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c18f71-3d79-44ea-879e-7f8a6ffded8d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:04.294613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 454/900] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 18:34:04 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:34:04.295464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c18f71-3d79-44ea-879e-7f8a6ffded8d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:04.425858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c18f71-3d79-44ea-879e-7f8a6ffded8d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:04.426201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ab733-b8cd-486e-9034-8155294f2baa None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:04.464449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:04.467833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecda78-196d-48fe-b025-5637a0f3d522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:04.468929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecda78-196d-48fe-b025-5637a0f3d522 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:04.469454 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecda78-196d-48fe-b025-5637a0f3d522 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:04.469939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecda78-196d-48fe-b025-5637a0f3d522 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:04.478528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda811e5-34c9-4663-94e3-98d195958862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:04.479151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda811e5-34c9-4663-94e3-98d195958862 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:04.479427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda811e5-34c9-4663-94e3-98d195958862 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:04.479706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda811e5-34c9-4663-94e3-98d195958862 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:04.480342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda811e5-34c9-4663-94e3-98d195958862 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:04.480834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda811e5-34c9-4663-94e3-98d195958862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:04.482173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecda78-196d-48fe-b025-5637a0f3d522 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:04.482502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda811e5-34c9-4663-94e3-98d195958862 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:04.483841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecda78-196d-48fe-b025-5637a0f3d522 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:04.488514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 455/901] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:34:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:04.495364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda811e5-34c9-4663-94e3-98d195958862 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-dcecda78-196d-48fe-b025-5637a0f3d522 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:34:04.498575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:04.499516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:04.500652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:04.500652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:04.500652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:34:04.500652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:34:04.500652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:34:04.500652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:34:04.500652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:34:04.500652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:34:04.500652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:34:04.500652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:04.501291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 447/902] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 18:34:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:04.507986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecda78-196d-48fe-b025-5637a0f3d522 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:04.534388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:04.623272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecda78-196d-48fe-b025-5637a0f3d522 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:04.643259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda811e5-34c9-4663-94e3-98d195958862 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:04.667181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f664d15-0791-46bd-8b32-af567946aff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:04.667805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f664d15-0791-46bd-8b32-af567946aff2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:04.668088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f664d15-0791-46bd-8b32-af567946aff2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:04.668364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f664d15-0791-46bd-8b32-af567946aff2 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:04.669135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f664d15-0791-46bd-8b32-af567946aff2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:04.669451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f664d15-0791-46bd-8b32-af567946aff2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:04.671414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f664d15-0791-46bd-8b32-af567946aff2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:04.687070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252e67f-44a8-464a-8b9c-8f3ee387993b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:04.687989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252e67f-44a8-464a-8b9c-8f3ee387993b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:04.688492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252e67f-44a8-464a-8b9c-8f3ee387993b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:04.688942 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252e67f-44a8-464a-8b9c-8f3ee387993b None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:04.689786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252e67f-44a8-464a-8b9c-8f3ee387993b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:04.690269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252e67f-44a8-464a-8b9c-8f3ee387993b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:04.691810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252e67f-44a8-464a-8b9c-8f3ee387993b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:04.699686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 448/903] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 18:34:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:04.699962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 456/904] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:34:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:04.705904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f664d15-0791-46bd-8b32-af567946aff2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:04.706028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252e67f-44a8-464a-8b9c-8f3ee387993b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:04.781331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:04.791463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:04.815775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f664d15-0791-46bd-8b32-af567946aff2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:04.842464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252e67f-44a8-464a-8b9c-8f3ee387993b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:04.857731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24784c93-ef51-48f6-a94f-28fe6300655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:04.858774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24784c93-ef51-48f6-a94f-28fe6300655f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:04.859253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24784c93-ef51-48f6-a94f-28fe6300655f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:04.859720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24784c93-ef51-48f6-a94f-28fe6300655f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:04.860630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24784c93-ef51-48f6-a94f-28fe6300655f None admin] RBAC: Adding query filter params (interface=public, service_id=67ddf76d6808459f932dfb56cb29fdad, region_id=RegionOne) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:04.861255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24784c93-ef51-48f6-a94f-28fe6300655f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:04.862981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24784c93-ef51-48f6-a94f-28fe6300655f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:04.869763 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 449/905] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 18:34:04 2022] GET /identity/v3/endpoints?service_id=67ddf76d6808459f932dfb56cb29fdad&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:04.881540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24784c93-ef51-48f6-a94f-28fe6300655f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:04.889462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c948c-7a95-446a-a363-491e9e776d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:04.890329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c948c-7a95-446a-a363-491e9e776d01 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:04.891242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c948c-7a95-446a-a363-491e9e776d01 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:04.891673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c948c-7a95-446a-a363-491e9e776d01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:04.984348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c948c-7a95-446a-a363-491e9e776d01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:04.986885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c948c-7a95-446a-a363-491e9e776d01 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:04.987782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24784c93-ef51-48f6-a94f-28fe6300655f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:05.035351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cb15b-b5e8-43dc-9e53-68f0e2c7fae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:05.036066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cb15b-b5e8-43dc-9e53-68f0e2c7fae7 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:05.036421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cb15b-b5e8-43dc-9e53-68f0e2c7fae7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:05.036766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cb15b-b5e8-43dc-9e53-68f0e2c7fae7 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:05.038947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cb15b-b5e8-43dc-9e53-68f0e2c7fae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:05.040845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cb15b-b5e8-43dc-9e53-68f0e2c7fae7 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:05.053602 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 450/906] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 18:34:04 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:34:05.063054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:05.114169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 457/907] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:34:04 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 18:34:05.122362 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cb15b-b5e8-43dc-9e53-68f0e2c7fae7 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:05.128800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:05.245480 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cb15b-b5e8-43dc-9e53-68f0e2c7fae7 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:05.286457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f35a4-35a7-4bc7-be76-f423a660d7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:05.287220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f35a4-35a7-4bc7-be76-f423a660d7ad None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:05.287569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f35a4-35a7-4bc7-be76-f423a660d7ad None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:05.287849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f35a4-35a7-4bc7-be76-f423a660d7ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:05.300051 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f35a4-35a7-4bc7-be76-f423a660d7ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:05.301560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f35a4-35a7-4bc7-be76-f423a660d7ad None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-6f0f35a4-35a7-4bc7-be76-f423a660d7ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:05.316671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:05.317816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:05.319414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:05.319414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:05.319414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:05.319414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 451/908] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:05.325862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c948c-7a95-446a-a363-491e9e776d01 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:05.432603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c948c-7a95-446a-a363-491e9e776d01 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:05.467904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8b567-69f2-4e8c-a16b-651553d0c718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:05.469177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8b567-69f2-4e8c-a16b-651553d0c718 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:05.469684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8b567-69f2-4e8c-a16b-651553d0c718 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:05.470179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8b567-69f2-4e8c-a16b-651553d0c718 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:05.471023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8b567-69f2-4e8c-a16b-651553d0c718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:05.471541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8b567-69f2-4e8c-a16b-651553d0c718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:05.473111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8b567-69f2-4e8c-a16b-651553d0c718 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:05.490126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 458/909] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:05.492428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f35a4-35a7-4bc7-be76-f423a660d7ad None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:05.556930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:05.612624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f35a4-35a7-4bc7-be76-f423a660d7ad None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:05.650674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85416b6-7a0c-419e-810a-fba098b2b3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:05.651668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85416b6-7a0c-419e-810a-fba098b2b3a8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:05.652153 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85416b6-7a0c-419e-810a-fba098b2b3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:05.652626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85416b6-7a0c-419e-810a-fba098b2b3a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:05.660129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85416b6-7a0c-419e-810a-fba098b2b3a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:05.661879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85416b6-7a0c-419e-810a-fba098b2b3a8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-c85416b6-7a0c-419e-810a-fba098b2b3a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:34:05.674337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:05.675279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:05.676262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:34:05.676262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:34:05.676262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:34:05.676262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:34:05.676262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:34:05.676262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:34:05.676262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:05.676262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:05.683800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 452/910] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:34:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:05.685449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8b567-69f2-4e8c-a16b-651553d0c718 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:05.706932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:05.793775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8b567-69f2-4e8c-a16b-651553d0c718 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:05.830422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd460615-8bb4-4712-9156-84eec1241d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:05.831382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd460615-8bb4-4712-9156-84eec1241d1b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:05.831866 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd460615-8bb4-4712-9156-84eec1241d1b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:05.832342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd460615-8bb4-4712-9156-84eec1241d1b None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:05.833281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd460615-8bb4-4712-9156-84eec1241d1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:05.833791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd460615-8bb4-4712-9156-84eec1241d1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:05.835977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd460615-8bb4-4712-9156-84eec1241d1b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:05.853424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 459/911] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:34:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:05.855848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85416b6-7a0c-419e-810a-fba098b2b3a8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:05.964745 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85416b6-7a0c-419e-810a-fba098b2b3a8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:05.983178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:06.001757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eaaa0a-5475-4679-951c-51805433a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:06.002741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eaaa0a-5475-4679-951c-51805433a188 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:06.003224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eaaa0a-5475-4679-951c-51805433a188 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:06.003726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eaaa0a-5475-4679-951c-51805433a188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:06.004661 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eaaa0a-5475-4679-951c-51805433a188 None admin] RBAC: Adding query filter params (role.id=8a81cd8421c8438cb3a40fd7e56a97eb, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=3038d1d505654ffb95b78fbc9005baf4) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:06.005195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eaaa0a-5475-4679-951c-51805433a188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:06.007463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eaaa0a-5475-4679-951c-51805433a188 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:06.019234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 453/912] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:34:05 2022] GET /identity/v3/role_assignments?user.id=3038d1d505654ffb95b78fbc9005baf4&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=8a81cd8421c8438cb3a40fd7e56a97eb => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:06.303795 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be4a63-8c56-43fe-b709-8964983ee467 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:06.304204 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be4a63-8c56-43fe-b709-8964983ee467 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:06.304386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be4a63-8c56-43fe-b709-8964983ee467 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:06.306054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 460/913] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:06.311603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f424c1-a310-48ca-9e42-5778c0314a77 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:06.311893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f424c1-a310-48ca-9e42-5778c0314a77 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:06.312173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f424c1-a310-48ca-9e42-5778c0314a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:06.331586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-80f424c1-a310-48ca-9e42-5778c0314a77 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:06.332795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-80f424c1-a310-48ca-9e42-5778c0314a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:06.404729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 454/914] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:06.583422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280515c-6a2f-4306-836c-81715e0d941d None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:06.583613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280515c-6a2f-4306-836c-81715e0d941d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:06.583900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280515c-6a2f-4306-836c-81715e0d941d None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:06.586085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 461/915] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:06.595508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ad6e2-84d3-4d25-a021-c922463e9e4b None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:06.595925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ad6e2-84d3-4d25-a021-c922463e9e4b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:06.596304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ad6e2-84d3-4d25-a021-c922463e9e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:06.618338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-d60ad6e2-84d3-4d25-a021-c922463e9e4b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:06.619540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d60ad6e2-84d3-4d25-a021-c922463e9e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:06.683325 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280515c-6a2f-4306-836c-81715e0d941d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:06.687687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 455/916] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:06.759307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:06.791774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280515c-6a2f-4306-836c-81715e0d941d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:06.827313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad19b6-5add-49d0-ade6-34f87c382d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:06.828671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad19b6-5add-49d0-ade6-34f87c382d7b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:06.829123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad19b6-5add-49d0-ade6-34f87c382d7b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:06.829561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad19b6-5add-49d0-ade6-34f87c382d7b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:06.830285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad19b6-5add-49d0-ade6-34f87c382d7b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:06.831792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad19b6-5add-49d0-ade6-34f87c382d7b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-9bad19b6-5add-49d0-ade6-34f87c382d7b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:06.841477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:06.842733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:06.844086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:06.846493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 462/917] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:34:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:06.853131 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ad6e2-84d3-4d25-a021-c922463e9e4b None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:06.925571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:06.967249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ad6e2-84d3-4d25-a021-c922463e9e4b None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:06.980992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad19b6-5add-49d0-ade6-34f87c382d7b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:07.009473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe7e2c-c8ef-410f-a13f-4452b981e045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:07.010863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe7e2c-c8ef-410f-a13f-4452b981e045 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:07.011370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe7e2c-c8ef-410f-a13f-4452b981e045 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:07.012089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe7e2c-c8ef-410f-a13f-4452b981e045 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:07.013163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe7e2c-c8ef-410f-a13f-4452b981e045 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:07.014039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe7e2c-c8ef-410f-a13f-4452b981e045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:07.015914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe7e2c-c8ef-410f-a13f-4452b981e045 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:07.022959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 456/918] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 18:34:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:07.027940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe7e2c-c8ef-410f-a13f-4452b981e045 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:07.092663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad19b6-5add-49d0-ade6-34f87c382d7b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:07.131770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ee9cd-e26f-44c4-91b8-4f1a76d131dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:07.132438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ee9cd-e26f-44c4-91b8-4f1a76d131dd None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:07.132713 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ee9cd-e26f-44c4-91b8-4f1a76d131dd None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:07.132981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ee9cd-e26f-44c4-91b8-4f1a76d131dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:07.133602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ee9cd-e26f-44c4-91b8-4f1a76d131dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:07.135754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ee9cd-e26f-44c4-91b8-4f1a76d131dd None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-886ee9cd-e26f-44c4-91b8-4f1a76d131dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:07.146139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:07.147743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 18:34:07.148591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:07.149921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:07.152953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 463/919] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:34:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:07.153960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ee9cd-e26f-44c4-91b8-4f1a76d131dd None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:07.166414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe7e2c-c8ef-410f-a13f-4452b981e045 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:07.170927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:07.206887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29294633-d693-4682-9392-93d5f9d36bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:07.207516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29294633-d693-4682-9392-93d5f9d36bd4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:07.207793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29294633-d693-4682-9392-93d5f9d36bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:07.208061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29294633-d693-4682-9392-93d5f9d36bd4 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:07.208684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29294633-d693-4682-9392-93d5f9d36bd4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:07.208982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29294633-d693-4682-9392-93d5f9d36bd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:07.211919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29294633-d693-4682-9392-93d5f9d36bd4 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:07.221410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 457/920] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 18:34:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:07.223614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29294633-d693-4682-9392-93d5f9d36bd4 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:07.295120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ee9cd-e26f-44c4-91b8-4f1a76d131dd None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:07.331696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29294633-d693-4682-9392-93d5f9d36bd4 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:07.349302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87567e66-7575-4e55-9898-df780f336e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:07.350580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87567e66-7575-4e55-9898-df780f336e40 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:07.351036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87567e66-7575-4e55-9898-df780f336e40 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:07.351777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87567e66-7575-4e55-9898-df780f336e40 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:07.352590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87567e66-7575-4e55-9898-df780f336e40 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:07.353081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87567e66-7575-4e55-9898-df780f336e40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:07.355043 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87567e66-7575-4e55-9898-df780f336e40 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:07.361809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 464/921] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:34:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:07.369157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87567e66-7575-4e55-9898-df780f336e40 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:07.375634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9507cd8-f815-43e3-91d8-9d0113c90601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:07.376244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9507cd8-f815-43e3-91d8-9d0113c90601 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:07.376520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9507cd8-f815-43e3-91d8-9d0113c90601 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:07.376793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9507cd8-f815-43e3-91d8-9d0113c90601 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:07.377529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9507cd8-f815-43e3-91d8-9d0113c90601 None admin] RBAC: Adding query filter params (interface=public, service_id=656cb08c9e7e4093b2e18a8082d4454f, region_id=RegionOne) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:07.377836 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9507cd8-f815-43e3-91d8-9d0113c90601 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:07.379197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9507cd8-f815-43e3-91d8-9d0113c90601 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:07.389328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 458/922] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 18:34:07 2022] GET /identity/v3/endpoints?service_id=656cb08c9e7e4093b2e18a8082d4454f&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:07.390805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9507cd8-f815-43e3-91d8-9d0113c90601 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:07.424077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:07.447271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:07.492137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87567e66-7575-4e55-9898-df780f336e40 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:07.520163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9507cd8-f815-43e3-91d8-9d0113c90601 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:07.527089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f49b7-cd31-41f8-adcd-41f54202ac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:07.527693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f49b7-cd31-41f8-adcd-41f54202ac73 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:07.527966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f49b7-cd31-41f8-adcd-41f54202ac73 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:07.528256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f49b7-cd31-41f8-adcd-41f54202ac73 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:07.528863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f49b7-cd31-41f8-adcd-41f54202ac73 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:07.529199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f49b7-cd31-41f8-adcd-41f54202ac73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:07.530545 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f49b7-cd31-41f8-adcd-41f54202ac73 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:07.536119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 465/923] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 8 18:34:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:07.537800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3991-894f-49c2-8cb5-e68bcb8e4a3d None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:07.538219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3991-894f-49c2-8cb5-e68bcb8e4a3d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:07.538496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3991-894f-49c2-8cb5-e68bcb8e4a3d None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:07.539920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 466/924] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:07.541857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a3991-894f-49c2-8cb5-e68bcb8e4a3d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:07.563034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046b6ba-2403-4c67-8bc1-924e5049760d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:07.563645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046b6ba-2403-4c67-8bc1-924e5049760d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:07.563932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046b6ba-2403-4c67-8bc1-924e5049760d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:07.564307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046b6ba-2403-4c67-8bc1-924e5049760d None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:07.565296 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046b6ba-2403-4c67-8bc1-924e5049760d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:07.566328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046b6ba-2403-4c67-8bc1-924e5049760d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:07.581085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 459/925] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 18:34:07 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:34:07.581085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e595fa-8be1-4b00-a9ff-c1e39c71f4ef None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:07.581085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e595fa-8be1-4b00-a9ff-c1e39c71f4ef None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:07.581085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e595fa-8be1-4b00-a9ff-c1e39c71f4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:07.606124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-53e595fa-8be1-4b00-a9ff-c1e39c71f4ef None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:07.608130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-53e595fa-8be1-4b00-a9ff-c1e39c71f4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:07.678215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a3991-894f-49c2-8cb5-e68bcb8e4a3d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:07.710736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:07.719006 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 460/926] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:07.726158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b810f6-afcc-4be3-989f-6bf35b34887f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:07.727148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b810f6-afcc-4be3-989f-6bf35b34887f None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:07.727619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b810f6-afcc-4be3-989f-6bf35b34887f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:07.728079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b810f6-afcc-4be3-989f-6bf35b34887f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:07.728973 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b810f6-afcc-4be3-989f-6bf35b34887f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:07.730616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b810f6-afcc-4be3-989f-6bf35b34887f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:07.801450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 467/927] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:34:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:34:07.998384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e595fa-8be1-4b00-a9ff-c1e39c71f4ef None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:08.028967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:08.108866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e595fa-8be1-4b00-a9ff-c1e39c71f4ef None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:08.153317 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb4f8c-2f2a-43de-86b7-01c76ab3a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:08.153981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4f8c-2f2a-43de-86b7-01c76ab3a04c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:08.154266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4f8c-2f2a-43de-86b7-01c76ab3a04c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:08.154541 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4f8c-2f2a-43de-86b7-01c76ab3a04c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:08.155141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb4f8c-2f2a-43de-86b7-01c76ab3a04c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:08.156610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb4f8c-2f2a-43de-86b7-01c76ab3a04c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-f6fb4f8c-2f2a-43de-86b7-01c76ab3a04c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:08.166935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:08.167967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:08.168873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 461/928] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 8 18:34:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:08.174534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b810f6-afcc-4be3-989f-6bf35b34887f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:08.268071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1a60d-7d4a-4846-8457-4c41cec80426 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:08.268277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1a60d-7d4a-4846-8457-4c41cec80426 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:08.268540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1a60d-7d4a-4846-8457-4c41cec80426 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:08.270183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 462/929] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:08.276017 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6d48f-0e11-4b99-86f2-0f6d72ef8d76 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:08.276310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6d48f-0e11-4b99-86f2-0f6d72ef8d76 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:08.276590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6d48f-0e11-4b99-86f2-0f6d72ef8d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:08.284312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b810f6-afcc-4be3-989f-6bf35b34887f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:08.295057 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-37b6d48f-0e11-4b99-86f2-0f6d72ef8d76 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:08.296261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-37b6d48f-0e11-4b99-86f2-0f6d72ef8d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:08.319544 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1c1f0-27eb-4131-9474-aa2919ee90dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:08.320192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1c1f0-27eb-4131-9474-aa2919ee90dc None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:08.320475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1c1f0-27eb-4131-9474-aa2919ee90dc None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:08.320746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1c1f0-27eb-4131-9474-aa2919ee90dc None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:08.321424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1c1f0-27eb-4131-9474-aa2919ee90dc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:08.321727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1c1f0-27eb-4131-9474-aa2919ee90dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:08.323089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1c1f0-27eb-4131-9474-aa2919ee90dc None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:08.329567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 468/930] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:34:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:08.335900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1c1f0-27eb-4131-9474-aa2919ee90dc None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:08.389140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 463/931] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:08.400354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:08.454247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1c1f0-27eb-4131-9474-aa2919ee90dc None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:08.494641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6505c3-5ca6-49b6-ba4a-f88bbc3ebf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:08.495613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6505c3-5ca6-49b6-ba4a-f88bbc3ebf05 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:08.496102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6505c3-5ca6-49b6-ba4a-f88bbc3ebf05 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:08.496583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6505c3-5ca6-49b6-ba4a-f88bbc3ebf05 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:08.497507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6505c3-5ca6-49b6-ba4a-f88bbc3ebf05 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:08.498020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6505c3-5ca6-49b6-ba4a-f88bbc3ebf05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:08.499576 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6505c3-5ca6-49b6-ba4a-f88bbc3ebf05 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:08.505209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 469/932] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:34:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:08.512738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6d48f-0e11-4b99-86f2-0f6d72ef8d76 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:08.545489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:08.628902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6d48f-0e11-4b99-86f2-0f6d72ef8d76 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:08.664489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a78761-42e9-4cd3-bbd5-a4412819afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:08.665178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a78761-42e9-4cd3-bbd5-a4412819afb8 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:08.665457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a78761-42e9-4cd3-bbd5-a4412819afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:08.666777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6505c3-5ca6-49b6-ba4a-f88bbc3ebf05 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:08.667132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a78761-42e9-4cd3-bbd5-a4412819afb8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:08.668059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a78761-42e9-4cd3-bbd5-a4412819afb8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:08.669686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a78761-42e9-4cd3-bbd5-a4412819afb8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:08.778395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6505c3-5ca6-49b6-ba4a-f88bbc3ebf05 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:08.783638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 464/933] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:34:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:34:08.803979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:08.842038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644d8ac-b6da-43f1-94ca-537956d63f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:08.843236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644d8ac-b6da-43f1-94ca-537956d63f5f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:08.844064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644d8ac-b6da-43f1-94ca-537956d63f5f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:08.844690 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644d8ac-b6da-43f1-94ca-537956d63f5f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:08.860518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644d8ac-b6da-43f1-94ca-537956d63f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-b644d8ac-b6da-43f1-94ca-537956d63f5f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:34:08.866491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:08.867540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:08.868395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:08.871497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 470/934] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 18:34:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:08.878375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a78761-42e9-4cd3-bbd5-a4412819afb8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:08.966140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:09.001086 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a78761-42e9-4cd3-bbd5-a4412819afb8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:09.037147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5151f2-a6c3-4e27-a44d-f79323cb9098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:09.039090 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5151f2-a6c3-4e27-a44d-f79323cb9098 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:09.039378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5151f2-a6c3-4e27-a44d-f79323cb9098 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:09.039656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5151f2-a6c3-4e27-a44d-f79323cb9098 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:09.040326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5151f2-a6c3-4e27-a44d-f79323cb9098 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:09.040633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5151f2-a6c3-4e27-a44d-f79323cb9098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:09.042035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5151f2-a6c3-4e27-a44d-f79323cb9098 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:09.050786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 465/935] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 18:34:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:09.056640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644d8ac-b6da-43f1-94ca-537956d63f5f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:09.160608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644d8ac-b6da-43f1-94ca-537956d63f5f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:09.196316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba6e5e-2981-4eec-82b9-b23bfc9d5da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:09.197037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba6e5e-2981-4eec-82b9-b23bfc9d5da4 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:09.197317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba6e5e-2981-4eec-82b9-b23bfc9d5da4 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:09.197576 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba6e5e-2981-4eec-82b9-b23bfc9d5da4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:09.208700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba6e5e-2981-4eec-82b9-b23bfc9d5da4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:09.210244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba6e5e-2981-4eec-82b9-b23bfc9d5da4 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-f9ba6e5e-2981-4eec-82b9-b23bfc9d5da4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:09.225820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:09.226830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:09.227719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:09.227719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:09.227719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:09.227719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 471/936] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:09.234291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5151f2-a6c3-4e27-a44d-f79323cb9098 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:09.342103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5151f2-a6c3-4e27-a44d-f79323cb9098 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:09.380772 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7b9ab-af52-4a2c-b38a-e1336d62596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:09.381959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7b9ab-af52-4a2c-b38a-e1336d62596e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:09.382469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7b9ab-af52-4a2c-b38a-e1336d62596e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:09.382942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7b9ab-af52-4a2c-b38a-e1336d62596e None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:09.383776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7b9ab-af52-4a2c-b38a-e1336d62596e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:09.384297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7b9ab-af52-4a2c-b38a-e1336d62596e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:09.385885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7b9ab-af52-4a2c-b38a-e1336d62596e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:09.387538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:09.400506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 466/937] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:09.409929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba6e5e-2981-4eec-82b9-b23bfc9d5da4 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:09.431070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:09.513590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba6e5e-2981-4eec-82b9-b23bfc9d5da4 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:09.549783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3413c-ccf6-47b4-853e-a5808ae840e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:09.550799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3413c-ccf6-47b4-853e-a5808ae840e2 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:09.551324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3413c-ccf6-47b4-853e-a5808ae840e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:09.551796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3413c-ccf6-47b4-853e-a5808ae840e2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:09.560225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3413c-ccf6-47b4-853e-a5808ae840e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:09.561940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3413c-ccf6-47b4-853e-a5808ae840e2 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-e8e3413c-ccf6-47b4-853e-a5808ae840e2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:34:09.578181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:09.579163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:09.580109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 472/938] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 18:34:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:09.587524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7b9ab-af52-4a2c-b38a-e1336d62596e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:09.706183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7b9ab-af52-4a2c-b38a-e1336d62596e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:09.746876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676e8dc-1cf8-44c0-9a18-6f82e3527909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:09.747878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676e8dc-1cf8-44c0-9a18-6f82e3527909 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:09.748404 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676e8dc-1cf8-44c0-9a18-6f82e3527909 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:09.748910 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676e8dc-1cf8-44c0-9a18-6f82e3527909 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:09.749892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676e8dc-1cf8-44c0-9a18-6f82e3527909 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:09.750412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676e8dc-1cf8-44c0-9a18-6f82e3527909 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:09.752071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676e8dc-1cf8-44c0-9a18-6f82e3527909 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:09.772856 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 467/939] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 18:34:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:09.778977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3413c-ccf6-47b4-853e-a5808ae840e2 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:09.849503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05af4d1-16f8-4087-83cf-2bb5ef7b9866 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:09.850758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05af4d1-16f8-4087-83cf-2bb5ef7b9866 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:09.851253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05af4d1-16f8-4087-83cf-2bb5ef7b9866 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:09.853027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 468/940] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:34:09.861079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d93ca-2007-443b-8701-df6af1cf53a9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:09.861312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d93ca-2007-443b-8701-df6af1cf53a9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:09.861591 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d93ca-2007-443b-8701-df6af1cf53a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:09.875953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:09.880501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-794d93ca-2007-443b-8701-df6af1cf53a9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:09.882398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-794d93ca-2007-443b-8701-df6af1cf53a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:09.901549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3413c-ccf6-47b4-853e-a5808ae840e2 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:09.946326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124a260-6b47-42ae-aa3d-4d36035f83ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:09.947548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:09.947846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124a260-6b47-42ae-aa3d-4d36035f83ae None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:09.948316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124a260-6b47-42ae-aa3d-4d36035f83ae None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:09.948958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124a260-6b47-42ae-aa3d-4d36035f83ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:09.962298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124a260-6b47-42ae-aa3d-4d36035f83ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:09.964247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124a260-6b47-42ae-aa3d-4d36035f83ae None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:09.971693 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 469/941] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-1124a260-6b47-42ae-aa3d-4d36035f83ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:09.985343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:09.986317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:09.987244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:09.987244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:09.987244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:09.989319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 473/942] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:09.995640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d93ca-2007-443b-8701-df6af1cf53a9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:10.106599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d93ca-2007-443b-8701-df6af1cf53a9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:10.144068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d017f7-7149-4e35-b22f-94313a80e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:10.145117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d017f7-7149-4e35-b22f-94313a80e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:10.145615 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d017f7-7149-4e35-b22f-94313a80e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:10.146084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d017f7-7149-4e35-b22f-94313a80e3f2 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:10.147478 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d017f7-7149-4e35-b22f-94313a80e3f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:10.148157 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d017f7-7149-4e35-b22f-94313a80e3f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:10.149750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d017f7-7149-4e35-b22f-94313a80e3f2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:10.161964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 470/943] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:10.167932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124a260-6b47-42ae-aa3d-4d36035f83ae None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:10.179028 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be7547-7529-43ff-a167-88b0833f4462 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:10.179428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be7547-7529-43ff-a167-88b0833f4462 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:10.179718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be7547-7529-43ff-a167-88b0833f4462 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:10.181378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 471/944] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:10.186573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fdaef-ed01-459d-9c66-c414176dedb1 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:10.186861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fdaef-ed01-459d-9c66-c414176dedb1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:10.187202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fdaef-ed01-459d-9c66-c414176dedb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:10.206597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-aa7fdaef-ed01-459d-9c66-c414176dedb1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:10.207972 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-aa7fdaef-ed01-459d-9c66-c414176dedb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:10.252819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:10.286945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 472/945] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:10.288286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fdaef-ed01-459d-9c66-c414176dedb1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:10.291300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124a260-6b47-42ae-aa3d-4d36035f83ae None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:10.321876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:10.326838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b01d54-dfb7-42f4-8473-49d8c0bfe674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:10.327644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b01d54-dfb7-42f4-8473-49d8c0bfe674 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:10.327922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b01d54-dfb7-42f4-8473-49d8c0bfe674 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:10.333343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b01d54-dfb7-42f4-8473-49d8c0bfe674 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:10.335695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b01d54-dfb7-42f4-8473-49d8c0bfe674 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:10.337205 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b01d54-dfb7-42f4-8473-49d8c0bfe674 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-a4b01d54-dfb7-42f4-8473-49d8c0bfe674 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:34:10.349080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:10.350141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:10.351474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:34:10.351474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:34:10.351474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:34:10.351474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:34:10.351474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:34:10.351474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:34:10.351474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:10.351474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:10.351474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 474/946] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:34:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:10.357377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b01d54-dfb7-42f4-8473-49d8c0bfe674 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:10.416795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fdaef-ed01-459d-9c66-c414176dedb1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:10.455403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d020ac-1d0b-45aa-b82c-b99d5e3d6c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:10.456264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d020ac-1d0b-45aa-b82c-b99d5e3d6c9d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:10.456703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d020ac-1d0b-45aa-b82c-b99d5e3d6c9d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:10.457219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d020ac-1d0b-45aa-b82c-b99d5e3d6c9d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:10.458077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d020ac-1d0b-45aa-b82c-b99d5e3d6c9d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:10.459740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d020ac-1d0b-45aa-b82c-b99d5e3d6c9d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:10.466497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b01d54-dfb7-42f4-8473-49d8c0bfe674 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-e9d020ac-1d0b-45aa-b82c-b99d5e3d6c9d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:10.469924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:10.471104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:10.472118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:10.479695 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 473/947] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:34:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:10.481197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d020ac-1d0b-45aa-b82c-b99d5e3d6c9d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:10.501929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bdad9-3d76-45e3-ac2a-bd28680e2506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:10.502537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bdad9-3d76-45e3-ac2a-bd28680e2506 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:10.502875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bdad9-3d76-45e3-ac2a-bd28680e2506 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:10.503151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bdad9-3d76-45e3-ac2a-bd28680e2506 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:10.503880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bdad9-3d76-45e3-ac2a-bd28680e2506 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:10.504187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bdad9-3d76-45e3-ac2a-bd28680e2506 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:10.505675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bdad9-3d76-45e3-ac2a-bd28680e2506 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:10.519082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 475/948] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:34:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:10.523747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:10.524950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bdad9-3d76-45e3-ac2a-bd28680e2506 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:10.586765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:10.609707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d020ac-1d0b-45aa-b82c-b99d5e3d6c9d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:10.644362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bdad9-3d76-45e3-ac2a-bd28680e2506 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:10.645925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b1553-b661-4c47-ae21-8966779ffac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:10.647413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1553-b661-4c47-ae21-8966779ffac9 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:10.647760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1553-b661-4c47-ae21-8966779ffac9 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:10.648171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1553-b661-4c47-ae21-8966779ffac9 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:10.648872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b1553-b661-4c47-ae21-8966779ffac9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:10.649273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b1553-b661-4c47-ae21-8966779ffac9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:10.650716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b1553-b661-4c47-ae21-8966779ffac9 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:10.660221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 474/949] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 18:34:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:10.661220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b1553-b661-4c47-ae21-8966779ffac9 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:10.681756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5166ab0-7b07-4da6-abf5-7f3c7f71d1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:10.682358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5166ab0-7b07-4da6-abf5-7f3c7f71d1ec None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:10.682689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5166ab0-7b07-4da6-abf5-7f3c7f71d1ec None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:10.682963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5166ab0-7b07-4da6-abf5-7f3c7f71d1ec None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/3038d1d505654ffb95b78fbc9005baf4/roles/8a81cd8421c8438cb3a40fd7e56a97eb` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:10.685575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5166ab0-7b07-4da6-abf5-7f3c7f71d1ec None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=3038d1d505654ffb95b78fbc9005baf4, role_id=8a81cd8421c8438cb3a40fd7e56a97eb)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:10.686987 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5166ab0-7b07-4da6-abf5-7f3c7f71d1ec None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:10.755301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 476/950] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:34:10 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/3038d1d505654ffb95b78fbc9005baf4/roles/8a81cd8421c8438cb3a40fd7e56a97eb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:34:10.757249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5166ab0-7b07-4da6-abf5-7f3c7f71d1ec None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:10.766897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:10.774487 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b1553-b661-4c47-ae21-8966779ffac9 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:10.816578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:10.817260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:10.817668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:10.817939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:10.818647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:10.820100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:10.823122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: WARNING keystone.server.flask.common [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 18:34:10.883359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5166ab0-7b07-4da6-abf5-7f3c7f71d1ec None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:10.897811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:10.900666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.sql.core [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 18:34:10.900666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 18:34:10.900666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [parameters: {'id': 'e8f4823ba51f45928034b3f323471067', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 18:34:10.900666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88172) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e8f4823ba51f45928034b3f323471067.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e8f4823ba51f45928034b3f323471067. Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application conn.query(q) Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 18:34:10.922990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application result.read() Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 08 18:34:10.924706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application next(self.gen) Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application yield resource Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application next(self.gen) Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise self.value Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application session.commit() Sep 08 18:34:10.927384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self.session.flush() Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application compat.raise_( Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 18:34:10.931026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 18:34:10.933715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application conn.query(q) Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application result.read() Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 18:34:10.935708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [parameters: {'id': 'e8f4823ba51f45928034b3f323471067', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:10.938100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:10.940078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:10.947844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:10.947844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 08 18:34:10.947844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 08 18:34:10.947844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:10.947844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:10.947844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 08 18:34:10.947844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 08 18:34:10.947844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e8f4823ba51f45928034b3f323471067. Sep 08 18:34:10.947844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:10.947844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 475/951] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Sep 8 18:34:10 2022] POST /identity/v3/projects => generated 262 bytes in 264 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:10.949318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4b1dd-ede9-42ff-91a5-23d6d600d063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:10.949318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4b1dd-ede9-42ff-91a5-23d6d600d063 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:10.949318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4b1dd-ede9-42ff-91a5-23d6d600d063 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:10.949318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4b1dd-ede9-42ff-91a5-23d6d600d063 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:10.951049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4b1dd-ede9-42ff-91a5-23d6d600d063 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:10.951049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4b1dd-ede9-42ff-91a5-23d6d600d063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:10.951330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:10.951663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4b1dd-ede9-42ff-91a5-23d6d600d063 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:10.955446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 477/952] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 18:34:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:10.962591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4b1dd-ede9-42ff-91a5-23d6d600d063 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:11.082232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bb746-e5ca-4f6e-8841-b5b6002ca0bb None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:11.097921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4b1dd-ede9-42ff-91a5-23d6d600d063 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:11.128514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfb771-f00b-4976-b568-5fbe5e566465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:11.129228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfb771-f00b-4976-b568-5fbe5e566465 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:11.129462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfb771-f00b-4976-b568-5fbe5e566465 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:11.129736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfb771-f00b-4976-b568-5fbe5e566465 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:11.138984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfb771-f00b-4976-b568-5fbe5e566465 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:11.140898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfb771-f00b-4976-b568-5fbe5e566465 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:11.142328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532b4a4-da66-4a5c-b20b-0b71aa5d161b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:11.143341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532b4a4-da66-4a5c-b20b-0b71aa5d161b None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:11.144109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532b4a4-da66-4a5c-b20b-0b71aa5d161b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:11.144588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532b4a4-da66-4a5c-b20b-0b71aa5d161b None admin] PATH_INFO: `/v3/endpoints` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:11.145493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532b4a4-da66-4a5c-b20b-0b71aa5d161b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:11.147706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532b4a4-da66-4a5c-b20b-0b71aa5d161b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-96bfb771-f00b-4976-b568-5fbe5e566465 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:34:11.154494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:11.155532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:11.156479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:34:11.156479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:34:11.156479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:34:11.156479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:34:11.156479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:34:11.156479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:34:11.156479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:11.156479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:11.156479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 476/953] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:34:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:11.161261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfb771-f00b-4976-b568-5fbe5e566465 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:11.182832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:11.230953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 478/954] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:34:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:34:11.232492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce747aef-ca5d-4372-b77a-08ad88895da4 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:11.232999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce747aef-ca5d-4372-b77a-08ad88895da4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:11.233449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce747aef-ca5d-4372-b77a-08ad88895da4 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:11.235072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 479/955] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:11.242359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac2df8-a49b-4b6b-927e-1423d60bc677 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:11.242760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac2df8-a49b-4b6b-927e-1423d60bc677 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:11.243170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac2df8-a49b-4b6b-927e-1423d60bc677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:11.266541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-79ac2df8-a49b-4b6b-927e-1423d60bc677 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:11.267978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-79ac2df8-a49b-4b6b-927e-1423d60bc677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:11.284065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfb771-f00b-4976-b568-5fbe5e566465 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:11.307730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:11.348312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495b6c7-cbdc-4422-a097-a295957f7b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:11.348942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495b6c7-cbdc-4422-a097-a295957f7b13 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:11.349245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495b6c7-cbdc-4422-a097-a295957f7b13 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:11.350254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495b6c7-cbdc-4422-a097-a295957f7b13 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:11.350944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495b6c7-cbdc-4422-a097-a295957f7b13 None admin] RBAC: Adding query filter params (name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:11.351446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495b6c7-cbdc-4422-a097-a295957f7b13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:11.353705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495b6c7-cbdc-4422-a097-a295957f7b13 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:11.363744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 480/956] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:11.383633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 477/957] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:34:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:11.638340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac2df8-a49b-4b6b-927e-1423d60bc677 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:11.747598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac2df8-a49b-4b6b-927e-1423d60bc677 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:11.788375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e0549-03e0-4f01-80ce-3389d90833a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:11.789319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e0549-03e0-4f01-80ce-3389d90833a7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:11.789771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e0549-03e0-4f01-80ce-3389d90833a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:11.790194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e0549-03e0-4f01-80ce-3389d90833a7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:11.790936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e0549-03e0-4f01-80ce-3389d90833a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:11.792472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e0549-03e0-4f01-80ce-3389d90833a7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-268e0549-03e0-4f01-80ce-3389d90833a7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:11.801968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:11.803975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 18:34:11.805998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:11.810250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 481/958] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:34:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:11.817413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495b6c7-cbdc-4422-a097-a295957f7b13 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:11.909211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:11.933540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495b6c7-cbdc-4422-a097-a295957f7b13 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:11.970020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6f6ba-4d68-450c-aea7-65dfdd452ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:11.970642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6f6ba-4d68-450c-aea7-65dfdd452ec3 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:11.970925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6f6ba-4d68-450c-aea7-65dfdd452ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:11.971201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6f6ba-4d68-450c-aea7-65dfdd452ec3 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:11.971847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6f6ba-4d68-450c-aea7-65dfdd452ec3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:11.972151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6f6ba-4d68-450c-aea7-65dfdd452ec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:11.973593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6f6ba-4d68-450c-aea7-65dfdd452ec3 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:11.979788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 478/959] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:34:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:11.985546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e0549-03e0-4f01-80ce-3389d90833a7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:12.016718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:12.096088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e0549-03e0-4f01-80ce-3389d90833a7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:12.133176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e4649-3b90-4c00-8b84-e2b2340f9186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:12.133792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4649-3b90-4c00-8b84-e2b2340f9186 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:12.134077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4649-3b90-4c00-8b84-e2b2340f9186 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:12.134360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4649-3b90-4c00-8b84-e2b2340f9186 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:12.134991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e4649-3b90-4c00-8b84-e2b2340f9186 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:12.135299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e4649-3b90-4c00-8b84-e2b2340f9186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:12.136664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e4649-3b90-4c00-8b84-e2b2340f9186 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:12.141875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 482/960] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:34:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:13.103476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da7066-c953-43d1-a291-511683b857d1 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:13.103999 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da7066-c953-43d1-a291-511683b857d1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:13.103999 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da7066-c953-43d1-a291-511683b857d1 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:13.105637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 479/961] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:13.111021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d7a0f-e202-4730-a888-081854764ea8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:13.111423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d7a0f-e202-4730-a888-081854764ea8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:13.111566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d7a0f-e202-4730-a888-081854764ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:13.131039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-6d2d7a0f-e202-4730-a888-081854764ea8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:13.132243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6d2d7a0f-e202-4730-a888-081854764ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:13.205845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 483/962] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:13.479670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da7066-c953-43d1-a291-511683b857d1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:13.532098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2256bf-b2a0-4e86-b12b-252caf79e050 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:13.532477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2256bf-b2a0-4e86-b12b-252caf79e050 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:13.532675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2256bf-b2a0-4e86-b12b-252caf79e050 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:13.536740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 484/963] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:13.541820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de709e8-44fa-43cb-b564-5200854906dd None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:13.542367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de709e8-44fa-43cb-b564-5200854906dd None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:13.543392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de709e8-44fa-43cb-b564-5200854906dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:13.567711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7de709e8-44fa-43cb-b564-5200854906dd None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:13.569605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7de709e8-44fa-43cb-b564-5200854906dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:13.588047 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:13.617797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da7066-c953-43d1-a291-511683b857d1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:13.654949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba5eaa-4929-4c7e-9fa8-2da0757ebde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:13.655624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba5eaa-4929-4c7e-9fa8-2da0757ebde8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:13.655906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba5eaa-4929-4c7e-9fa8-2da0757ebde8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:13.656178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba5eaa-4929-4c7e-9fa8-2da0757ebde8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:13.658499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 485/964] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:13.666659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:13.677002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba5eaa-4929-4c7e-9fa8-2da0757ebde8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-85ba5eaa-4929-4c7e-9fa8-2da0757ebde8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:34:13.682849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:13.683831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:13.684831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:13.687564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 480/965] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 18:34:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:13.694719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de709e8-44fa-43cb-b564-5200854906dd None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:13.776369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3045b5-93fe-4260-9b8f-e069f452b5b8 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:13.777816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3045b5-93fe-4260-9b8f-e069f452b5b8 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:13.778206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3045b5-93fe-4260-9b8f-e069f452b5b8 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:13.780257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 481/966] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:13.785380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705c1c0-ae6c-461f-9257-5a655e067e1d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:13.785861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705c1c0-ae6c-461f-9257-5a655e067e1d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:13.786163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705c1c0-ae6c-461f-9257-5a655e067e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:13.806368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de709e8-44fa-43cb-b564-5200854906dd None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:13.810467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-8705c1c0-ae6c-461f-9257-5a655e067e1d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:13.812121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8705c1c0-ae6c-461f-9257-5a655e067e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:13.847812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb746b-1abd-4440-9542-830393f8361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:13.848719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb746b-1abd-4440-9542-830393f8361e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:13.849277 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb746b-1abd-4440-9542-830393f8361e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:13.849751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb746b-1abd-4440-9542-830393f8361e None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:13.850735 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb746b-1abd-4440-9542-830393f8361e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:13.851207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb746b-1abd-4440-9542-830393f8361e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:13.852947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb746b-1abd-4440-9542-830393f8361e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:13.862313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 486/967] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 18:34:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:13.874066 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb746b-1abd-4440-9542-830393f8361e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:13.894028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:13.906388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 482/968] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:13.954767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705c1c0-ae6c-461f-9257-5a655e067e1d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:13.991647 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb746b-1abd-4440-9542-830393f8361e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.028244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e2fa1-2977-4469-a47a-e4fdd40f40ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.029148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e2fa1-2977-4469-a47a-e4fdd40f40ea None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.029580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e2fa1-2977-4469-a47a-e4fdd40f40ea None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.030030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e2fa1-2977-4469-a47a-e4fdd40f40ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.040750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e2fa1-2977-4469-a47a-e4fdd40f40ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.042536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e2fa1-2977-4469-a47a-e4fdd40f40ea None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-131e2fa1-2977-4469-a47a-e4fdd40f40ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:14.058543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:14.059714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:14.060818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:14.060818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:14.060818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:14.062562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 487/969] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:14.068519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e2fa1-2977-4469-a47a-e4fdd40f40ea None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:14.072074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705c1c0-ae6c-461f-9257-5a655e067e1d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.110826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:14.123418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468bdf4-bd30-4322-a7e9-87171e4aeb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.124396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468bdf4-bd30-4322-a7e9-87171e4aeb25 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.124728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468bdf4-bd30-4322-a7e9-87171e4aeb25 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.125199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468bdf4-bd30-4322-a7e9-87171e4aeb25 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.126368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468bdf4-bd30-4322-a7e9-87171e4aeb25 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.128191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468bdf4-bd30-4322-a7e9-87171e4aeb25 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:14.136140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-d468bdf4-bd30-4322-a7e9-87171e4aeb25 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:14.140558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:14.141853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:14.142891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:14.151542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 483/970] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:34:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:14.153841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468bdf4-bd30-4322-a7e9-87171e4aeb25 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:14.188304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e2fa1-2977-4469-a47a-e4fdd40f40ea None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.228080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d52a5-4791-4cb8-a5c1-ce06f91a64ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.228689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d52a5-4791-4cb8-a5c1-ce06f91a64ff None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.228975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d52a5-4791-4cb8-a5c1-ce06f91a64ff None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.229280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d52a5-4791-4cb8-a5c1-ce06f91a64ff None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.229917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d52a5-4791-4cb8-a5c1-ce06f91a64ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:14.230240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d52a5-4791-4cb8-a5c1-ce06f91a64ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.231582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d52a5-4791-4cb8-a5c1-ce06f91a64ff None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:14.249409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 488/971] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:14.250396 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d52a5-4791-4cb8-a5c1-ce06f91a64ff None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:14.275077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468bdf4-bd30-4322-a7e9-87171e4aeb25 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.314173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848437d-1fa7-435c-90f7-edf8e65fe86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.314783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848437d-1fa7-435c-90f7-edf8e65fe86e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.315064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848437d-1fa7-435c-90f7-edf8e65fe86e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.315339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848437d-1fa7-435c-90f7-edf8e65fe86e None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.315966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848437d-1fa7-435c-90f7-edf8e65fe86e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:14.316273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848437d-1fa7-435c-90f7-edf8e65fe86e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.317663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848437d-1fa7-435c-90f7-edf8e65fe86e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:14.322558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 484/972] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 18:34:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:14.323565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848437d-1fa7-435c-90f7-edf8e65fe86e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:14.373832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d52a5-4791-4cb8-a5c1-ce06f91a64ff None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.384159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:14.398719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:14.409111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.409716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.409989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.410259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.410915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.413620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:14.415170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WARNING keystone.server.flask.common [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 18:34:14.431249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848437d-1fa7-435c-90f7-edf8e65fe86e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.466635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497850d5-9830-4daf-94f6-5e1feaa11281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.467259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497850d5-9830-4daf-94f6-5e1feaa11281 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.467537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497850d5-9830-4daf-94f6-5e1feaa11281 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.467814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497850d5-9830-4daf-94f6-5e1feaa11281 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.468921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.sql.core [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 18:34:14.468921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 18:34:14.468921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [parameters: {'user_id': '7af5bb8c7c224e7fa8890772e2353d8e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 18:34:14.468921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88173) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 18:34:14.475916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497850d5-9830-4daf-94f6-5e1feaa11281 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application conn.query(q) Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application result.read() Sep 08 18:34:14.476086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 18:34:14.477137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application next(self.gen) Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application yield resource Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application next(self.gen) Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise self.value Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application session.commit() Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 18:34:14.483515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self.session.flush() Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application compat.raise_( Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 18:34:14.484430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 18:34:14.485364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application conn.query(q) Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application result.read() Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 18:34:14.486223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [parameters: {'user_id': '7af5bb8c7c224e7fa8890772e2353d8e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:14.487083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 08 18:34:14.487925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 08 18:34:14.488902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 08 18:34:14.488902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 08 18:34:14.488902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 18:34:14.488902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 18:34:14.488902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 18:34:14.488902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:14.489546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497850d5-9830-4daf-94f6-5e1feaa11281 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:14.489636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 489/973] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 8 18:34:14 2022] POST /identity/v3/users => generated 160 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:14.489636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-497850d5-9830-4daf-94f6-5e1feaa11281 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:34:14.489766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:14.497575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 485/974] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 18:34:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b94c05-0558-41c3-80fe-1f94264b90e9 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.499568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b94c05-0558-41c3-80fe-1f94264b90e9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.500532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b94c05-0558-41c3-80fe-1f94264b90e9 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.500532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 486/975] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:14.500532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b94c05-0558-41c3-80fe-1f94264b90e9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:14.594347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e20cc-673d-4afa-8e25-b3ad66df319e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.600681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b94c05-0558-41c3-80fe-1f94264b90e9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.619044 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:14.630720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b63cb6-a4bc-44d3-a937-bdd497960348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.631883 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63cb6-a4bc-44d3-a937-bdd497960348 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.632159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63cb6-a4bc-44d3-a937-bdd497960348 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.632414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63cb6-a4bc-44d3-a937-bdd497960348 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.633121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b63cb6-a4bc-44d3-a937-bdd497960348 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:14.633424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b63cb6-a4bc-44d3-a937-bdd497960348 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.635017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b63cb6-a4bc-44d3-a937-bdd497960348 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:14.639269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8cef7-0683-4663-ae24-61ddbe062c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.640074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8cef7-0683-4663-ae24-61ddbe062c97 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.640508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8cef7-0683-4663-ae24-61ddbe062c97 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.640827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8cef7-0683-4663-ae24-61ddbe062c97 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.643700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 490/976] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 18:34:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:14.644507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b63cb6-a4bc-44d3-a937-bdd497960348 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:14.648488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8cef7-0683-4663-ae24-61ddbe062c97 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.649959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8cef7-0683-4663-ae24-61ddbe062c97 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-b3c8cef7-0683-4663-ae24-61ddbe062c97 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:34:14.661488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:14.662676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:14.663853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 487/977] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:34:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:14.664959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f1376-cb3b-40c8-bdb6-00822a1eed0c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.664959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f1376-cb3b-40c8-bdb6-00822a1eed0c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.664959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f1376-cb3b-40c8-bdb6-00822a1eed0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.682398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b43f1376-cb3b-40c8-bdb6-00822a1eed0c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:14.683587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b43f1376-cb3b-40c8-bdb6-00822a1eed0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:14.698783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:14.749566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b63cb6-a4bc-44d3-a937-bdd497960348 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.750016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 488/978] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:14.750843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f1376-cb3b-40c8-bdb6-00822a1eed0c None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:14.785866 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7cc85-16ba-4e10-8e9a-68627c4303e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.786474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7cc85-16ba-4e10-8e9a-68627c4303e0 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.786790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7cc85-16ba-4e10-8e9a-68627c4303e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.787016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7cc85-16ba-4e10-8e9a-68627c4303e0 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.788298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7cc85-16ba-4e10-8e9a-68627c4303e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:14.788602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7cc85-16ba-4e10-8e9a-68627c4303e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.790033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7cc85-16ba-4e10-8e9a-68627c4303e0 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:14.812117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 491/979] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 8 18:34:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:14.812117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7cc85-16ba-4e10-8e9a-68627c4303e0 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:14.862945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f1376-cb3b-40c8-bdb6-00822a1eed0c None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.865722 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:14.900261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3a918-c1a6-42ac-b98c-ffd45c7d8385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.900861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3a918-c1a6-42ac-b98c-ffd45c7d8385 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.901173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3a918-c1a6-42ac-b98c-ffd45c7d8385 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.901448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3a918-c1a6-42ac-b98c-ffd45c7d8385 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.902233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3a918-c1a6-42ac-b98c-ffd45c7d8385 None admin] RBAC: Adding query filter params (interface=admin, service_id=656cb08c9e7e4093b2e18a8082d4454f, region_id=RegionOne) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:14.902464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3a918-c1a6-42ac-b98c-ffd45c7d8385 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.903882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3a918-c1a6-42ac-b98c-ffd45c7d8385 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:14.909401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 489/980] 10.222.0.25 () {56 vars in 1249 bytes} [Thu Sep 8 18:34:14 2022] GET /identity/v3/endpoints?service_id=656cb08c9e7e4093b2e18a8082d4454f&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:14.910258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3a918-c1a6-42ac-b98c-ffd45c7d8385 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:14.926458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7cc85-16ba-4e10-8e9a-68627c4303e0 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:14.937510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:14.965279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2adde-10e6-43ac-b3dc-ab08e419af3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:14.965891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2adde-10e6-43ac-b3dc-ab08e419af3f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:14.966652 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2adde-10e6-43ac-b3dc-ab08e419af3f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:14.966938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2adde-10e6-43ac-b3dc-ab08e419af3f None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:14.967676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2adde-10e6-43ac-b3dc-ab08e419af3f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:14.967983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2adde-10e6-43ac-b3dc-ab08e419af3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:14.969693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2adde-10e6-43ac-b3dc-ab08e419af3f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:15.005546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 492/981] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 8 18:34:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:15.007017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2adde-10e6-43ac-b3dc-ab08e419af3f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:15.027535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3a918-c1a6-42ac-b98c-ffd45c7d8385 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:15.064780 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd0a2e-03bd-4f29-aa7a-4dc760537923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:15.065432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd0a2e-03bd-4f29-aa7a-4dc760537923 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:15.065710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd0a2e-03bd-4f29-aa7a-4dc760537923 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:15.065979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd0a2e-03bd-4f29-aa7a-4dc760537923 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:15.108667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:15.137528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2adde-10e6-43ac-b3dc-ab08e419af3f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:15.147585 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd0a2e-03bd-4f29-aa7a-4dc760537923 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:15.149269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd0a2e-03bd-4f29-aa7a-4dc760537923 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:15.174474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a35cf-36ae-41e2-ad9a-c8fa3c28c509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:15.175088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a35cf-36ae-41e2-ad9a-c8fa3c28c509 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:15.175371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a35cf-36ae-41e2-ad9a-c8fa3c28c509 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:15.175637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a35cf-36ae-41e2-ad9a-c8fa3c28c509 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:15.176246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a35cf-36ae-41e2-ad9a-c8fa3c28c509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:15.177649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a35cf-36ae-41e2-ad9a-c8fa3c28c509 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:15.184136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 493/982] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 18:34:15 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:34:15.184955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a35cf-36ae-41e2-ad9a-c8fa3c28c509 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:15.222937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:15.305273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 490/983] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:34:14 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:34:15.311352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd0a2e-03bd-4f29-aa7a-4dc760537923 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:15.337277 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a35cf-36ae-41e2-ad9a-c8fa3c28c509 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:15.379908 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca97065-2536-4435-a783-973b0ff8aaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:15.380520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca97065-2536-4435-a783-973b0ff8aaa6 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:15.380795 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca97065-2536-4435-a783-973b0ff8aaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:15.381191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca97065-2536-4435-a783-973b0ff8aaa6 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:15.382364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca97065-2536-4435-a783-973b0ff8aaa6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:15.383479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca97065-2536-4435-a783-973b0ff8aaa6 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:15.401505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:15.432349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd0a2e-03bd-4f29-aa7a-4dc760537923 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:15.465957 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 494/984] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:34:15 2022] POST /identity/v3/services => generated 242 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:34:15.473642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1cf6d-486c-4ab3-a40a-a387884d9793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:15.474303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1cf6d-486c-4ab3-a40a-a387884d9793 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:15.474611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1cf6d-486c-4ab3-a40a-a387884d9793 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:15.474954 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1cf6d-486c-4ab3-a40a-a387884d9793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:15.487756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1cf6d-486c-4ab3-a40a-a387884d9793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:15.489282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1cf6d-486c-4ab3-a40a-a387884d9793 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-a9a1cf6d-486c-4ab3-a40a-a387884d9793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:15.508394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:15.509714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:15.511193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:15.511193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:15.511193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:15.511193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 491/985] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:15.515482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca97065-2536-4435-a783-973b0ff8aaa6 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:15.623770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:15.635779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca97065-2536-4435-a783-973b0ff8aaa6 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:15.675281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568425f-afff-4d32-91a0-a160efc40d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:15.675837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568425f-afff-4d32-91a0-a160efc40d61 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:15.676119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568425f-afff-4d32-91a0-a160efc40d61 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:15.676394 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568425f-afff-4d32-91a0-a160efc40d61 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:15.677069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568425f-afff-4d32-91a0-a160efc40d61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:15.677429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568425f-afff-4d32-91a0-a160efc40d61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:15.679749 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568425f-afff-4d32-91a0-a160efc40d61 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:15.692648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 495/986] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:15.698547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1cf6d-486c-4ab3-a40a-a387884d9793 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:15.802554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1cf6d-486c-4ab3-a40a-a387884d9793 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:15.821625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:15.839805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa47396-3fa1-4663-bb6f-3a687c38b1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:15.840530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa47396-3fa1-4663-bb6f-3a687c38b1ac None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:15.840888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa47396-3fa1-4663-bb6f-3a687c38b1ac None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:15.841267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa47396-3fa1-4663-bb6f-3a687c38b1ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:15.848482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa47396-3fa1-4663-bb6f-3a687c38b1ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:15.850634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa47396-3fa1-4663-bb6f-3a687c38b1ac None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-2aa47396-3fa1-4663-bb6f-3a687c38b1ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:34:15.869645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:15.870821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:15.871949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:34:15.871949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:34:15.871949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:34:15.871949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:34:15.871949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:34:15.871949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:34:15.871949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:15.871949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:15.872621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 492/987] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:34:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:15.877168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568425f-afff-4d32-91a0-a160efc40d61 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:15.983810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568425f-afff-4d32-91a0-a160efc40d61 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:16.018901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dade0-490f-460e-ba30-1599251b0c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:16.019513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dade0-490f-460e-ba30-1599251b0c25 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:16.019808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dade0-490f-460e-ba30-1599251b0c25 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:16.020071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dade0-490f-460e-ba30-1599251b0c25 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:16.020757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dade0-490f-460e-ba30-1599251b0c25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:16.021088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dade0-490f-460e-ba30-1599251b0c25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:16.022634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dade0-490f-460e-ba30-1599251b0c25 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:16.040268 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 496/988] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:34:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:16.041550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa47396-3fa1-4663-bb6f-3a687c38b1ac None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:16.147998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa47396-3fa1-4663-bb6f-3a687c38b1ac None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:16.190788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602e51e-630d-4867-bdbf-64a4848b38d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:16.191462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602e51e-630d-4867-bdbf-64a4848b38d2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:16.191792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602e51e-630d-4867-bdbf-64a4848b38d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:16.192065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602e51e-630d-4867-bdbf-64a4848b38d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:16.192875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602e51e-630d-4867-bdbf-64a4848b38d2 None admin] RBAC: Adding query filter params (role.id=8a81cd8421c8438cb3a40fd7e56a97eb, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=3038d1d505654ffb95b78fbc9005baf4) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:16.193207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602e51e-630d-4867-bdbf-64a4848b38d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:16.194671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602e51e-630d-4867-bdbf-64a4848b38d2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:16.213242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 493/989] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:34:16 2022] GET /identity/v3/role_assignments?user.id=3038d1d505654ffb95b78fbc9005baf4&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=8a81cd8421c8438cb3a40fd7e56a97eb => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:17.420769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e1997-dfb1-4da4-850f-3383b6034217 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:17.421333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e1997-dfb1-4da4-850f-3383b6034217 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:17.421333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e1997-dfb1-4da4-850f-3383b6034217 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:17.422691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 497/990] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:17.428231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842fccd-17f7-4277-98dc-acccb824183c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:17.428810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842fccd-17f7-4277-98dc-acccb824183c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:17.428810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842fccd-17f7-4277-98dc-acccb824183c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:17.447775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-4842fccd-17f7-4277-98dc-acccb824183c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:17.449119 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4842fccd-17f7-4277-98dc-acccb824183c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:17.484303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:17.518265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 494/991] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:17.662254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996b8f2-b4e9-4b42-b12c-c91e3e982c08 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:17.662567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996b8f2-b4e9-4b42-b12c-c91e3e982c08 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:17.662799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996b8f2-b4e9-4b42-b12c-c91e3e982c08 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:17.664291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 498/992] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:17.669385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52cb59-e067-4008-adc6-6c909bb2f1e5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:17.669672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52cb59-e067-4008-adc6-6c909bb2f1e5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:17.669942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52cb59-e067-4008-adc6-6c909bb2f1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:17.688585 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ae52cb59-e067-4008-adc6-6c909bb2f1e5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:17.689715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ae52cb59-e067-4008-adc6-6c909bb2f1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:17.756873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 495/993] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:17.785912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996b8f2-b4e9-4b42-b12c-c91e3e982c08 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:17.799480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:17.902435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996b8f2-b4e9-4b42-b12c-c91e3e982c08 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:17.938764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015616c9-2941-41fe-a6b1-d33a5deb16dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:17.939474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015616c9-2941-41fe-a6b1-d33a5deb16dc None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:17.939760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015616c9-2941-41fe-a6b1-d33a5deb16dc None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:17.940037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015616c9-2941-41fe-a6b1-d33a5deb16dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:17.940669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015616c9-2941-41fe-a6b1-d33a5deb16dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:17.942081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015616c9-2941-41fe-a6b1-d33a5deb16dc None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-015616c9-2941-41fe-a6b1-d33a5deb16dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:17.952107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:17.953233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:17.958115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 499/994] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 8 18:34:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:17.960921 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52cb59-e067-4008-adc6-6c909bb2f1e5 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:18.030951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015616c9-2941-41fe-a6b1-d33a5deb16dc None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:18.075351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:18.080067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52cb59-e067-4008-adc6-6c909bb2f1e5 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:18.117175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:18.124792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d880a-018c-491d-9c46-0d9e6dd96139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:18.125308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d880a-018c-491d-9c46-0d9e6dd96139 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:18.125581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d880a-018c-491d-9c46-0d9e6dd96139 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:18.125866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d880a-018c-491d-9c46-0d9e6dd96139 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:18.126513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d880a-018c-491d-9c46-0d9e6dd96139 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:18.126820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d880a-018c-491d-9c46-0d9e6dd96139 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:18.133212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d880a-018c-491d-9c46-0d9e6dd96139 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:18.134213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 496/995] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 18:34:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:18.139971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d880a-018c-491d-9c46-0d9e6dd96139 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:18.164580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015616c9-2941-41fe-a6b1-d33a5deb16dc None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:18.210633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4456050-6c17-49db-a1ca-45c1c8caa3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:18.211193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4456050-6c17-49db-a1ca-45c1c8caa3f1 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:18.211483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4456050-6c17-49db-a1ca-45c1c8caa3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:18.211762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4456050-6c17-49db-a1ca-45c1c8caa3f1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:18.212552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4456050-6c17-49db-a1ca-45c1c8caa3f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:18.213951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4456050-6c17-49db-a1ca-45c1c8caa3f1 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-e4456050-6c17-49db-a1ca-45c1c8caa3f1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:18.223966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:18.225810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 18:34:18.226716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:18.231162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 500/996] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:34:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:18.234288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4456050-6c17-49db-a1ca-45c1c8caa3f1 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:18.277167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d880a-018c-491d-9c46-0d9e6dd96139 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:18.321403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c899f21-4885-4618-889f-b34867159267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:18.322374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c899f21-4885-4618-889f-b34867159267 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:18.322858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c899f21-4885-4618-889f-b34867159267 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:18.323325 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c899f21-4885-4618-889f-b34867159267 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:18.324399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c899f21-4885-4618-889f-b34867159267 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:18.324896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c899f21-4885-4618-889f-b34867159267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:18.327030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c899f21-4885-4618-889f-b34867159267 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:18.332556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 497/997] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 8 18:34:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:18.340630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c899f21-4885-4618-889f-b34867159267 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:18.377852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4456050-6c17-49db-a1ca-45c1c8caa3f1 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:18.384727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:18.423303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764196ad-54e5-4a83-aad8-a5c617113352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:18.424023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764196ad-54e5-4a83-aad8-a5c617113352 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:18.424193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764196ad-54e5-4a83-aad8-a5c617113352 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:18.424461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764196ad-54e5-4a83-aad8-a5c617113352 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:18.425269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764196ad-54e5-4a83-aad8-a5c617113352 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:18.425821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764196ad-54e5-4a83-aad8-a5c617113352 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:18.428312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764196ad-54e5-4a83-aad8-a5c617113352 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:18.434760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 501/998] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:34:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:18.440883 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764196ad-54e5-4a83-aad8-a5c617113352 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:18.444671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:18.507339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c899f21-4885-4618-889f-b34867159267 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:18.555577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9cebd-14bb-495e-9f79-e1768fe30a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:18.556204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9cebd-14bb-495e-9f79-e1768fe30a54 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:18.556708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9cebd-14bb-495e-9f79-e1768fe30a54 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:18.557229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9cebd-14bb-495e-9f79-e1768fe30a54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:18.558344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9cebd-14bb-495e-9f79-e1768fe30a54 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:18.560726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9cebd-14bb-495e-9f79-e1768fe30a54 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:18.563977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764196ad-54e5-4a83-aad8-a5c617113352 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:18.610593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3d2bf-6a5f-4659-9595-8794545c98ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:18.611199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3d2bf-6a5f-4659-9595-8794545c98ea None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:18.611476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3d2bf-6a5f-4659-9595-8794545c98ea None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:18.611745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3d2bf-6a5f-4659-9595-8794545c98ea None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:18.612375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3d2bf-6a5f-4659-9595-8794545c98ea None admin] RBAC: Adding query filter params (type=compute) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:18.612675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3d2bf-6a5f-4659-9595-8794545c98ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:18.614087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3d2bf-6a5f-4659-9595-8794545c98ea None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:18.622635 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 502/999] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:34:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:18.623863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871dbcf-e42a-4ac5-ad81-868038343ed9 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:18.624141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871dbcf-e42a-4ac5-ad81-868038343ed9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:18.624412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871dbcf-e42a-4ac5-ad81-868038343ed9 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:18.625652 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 503/1000] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:18.630378 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871dbcf-e42a-4ac5-ad81-868038343ed9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:18.638818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 498/1001] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:34:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:34:18.640412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a1a8f-4212-405a-8812-f8fc549369cf None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:18.640870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a1a8f-4212-405a-8812-f8fc549369cf None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:18.641177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a1a8f-4212-405a-8812-f8fc549369cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:18.671464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-905a1a8f-4212-405a-8812-f8fc549369cf None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:18.672840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-905a1a8f-4212-405a-8812-f8fc549369cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:18.738403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:18.777180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 499/1002] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:18 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:18.798499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871dbcf-e42a-4ac5-ad81-868038343ed9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:18.851596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5882f-f804-448f-977b-05fd6798a0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:18.852217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5882f-f804-448f-977b-05fd6798a0de None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:18.852500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5882f-f804-448f-977b-05fd6798a0de None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:18.852777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5882f-f804-448f-977b-05fd6798a0de None admin] PATH_INFO: `/v3/endpoints` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:18.853523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5882f-f804-448f-977b-05fd6798a0de None admin] RBAC: Adding query filter params (interface=public, service_id=c6598c15d6054a23a431157b7cd62cf5, region_id=RegionOne) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:18.853834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5882f-f804-448f-977b-05fd6798a0de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:18.855204 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5882f-f804-448f-977b-05fd6798a0de None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:18.860860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 504/1003] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 18:34:18 2022] GET /identity/v3/endpoints?service_id=c6598c15d6054a23a431157b7cd62cf5&interface=public®ion_id=RegionOne => generated 194 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:18.867152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a1a8f-4212-405a-8812-f8fc549369cf None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:18.888153 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:18.994268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a1a8f-4212-405a-8812-f8fc549369cf None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:19.040225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5862afd-7e88-444a-aa06-737a35999382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:19.040839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5862afd-7e88-444a-aa06-737a35999382 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:19.041143 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5862afd-7e88-444a-aa06-737a35999382 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:19.041426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5862afd-7e88-444a-aa06-737a35999382 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:19.042073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5862afd-7e88-444a-aa06-737a35999382 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:19.043472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5862afd-7e88-444a-aa06-737a35999382 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:19.051689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 500/1004] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 18:34:18 2022] GET /identity/v3/services => generated 1727 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:34:19.074707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5882f-f804-448f-977b-05fd6798a0de None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:19.195585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5882f-f804-448f-977b-05fd6798a0de None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:19.227293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:19.233464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502c5d5-7482-4ad5-9beb-9b81793954af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:19.234106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502c5d5-7482-4ad5-9beb-9b81793954af None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:19.234387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502c5d5-7482-4ad5-9beb-9b81793954af None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:19.234658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502c5d5-7482-4ad5-9beb-9b81793954af None admin] PATH_INFO: `/v3/services/image` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:19.235250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502c5d5-7482-4ad5-9beb-9b81793954af None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:19.236605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502c5d5-7482-4ad5-9beb-9b81793954af None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-8502c5d5-7482-4ad5-9beb-9b81793954af None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:19.246173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:19.247372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:19.248391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 505/1005] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 8 18:34:19 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:19.253341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5862afd-7e88-444a-aa06-737a35999382 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:19.337326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:19.360026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5862afd-7e88-444a-aa06-737a35999382 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:19.397156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56f443-9fa5-4a32-b4cd-a8adcabf358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:19.397775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56f443-9fa5-4a32-b4cd-a8adcabf358b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:19.398060 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56f443-9fa5-4a32-b4cd-a8adcabf358b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:19.398339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56f443-9fa5-4a32-b4cd-a8adcabf358b None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:19.398976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56f443-9fa5-4a32-b4cd-a8adcabf358b None admin] RBAC: Adding query filter params (name=image) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:19.399283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56f443-9fa5-4a32-b4cd-a8adcabf358b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:19.400657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56f443-9fa5-4a32-b4cd-a8adcabf358b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:19.406197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 501/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:19.411992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502c5d5-7482-4ad5-9beb-9b81793954af None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:19.516009 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502c5d5-7482-4ad5-9beb-9b81793954af None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:19.551349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406f354-3119-461f-bdc6-0194ef296b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:19.551963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406f354-3119-461f-bdc6-0194ef296b75 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:19.552246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406f354-3119-461f-bdc6-0194ef296b75 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:19.552521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406f354-3119-461f-bdc6-0194ef296b75 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:19.553772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406f354-3119-461f-bdc6-0194ef296b75 None admin] RBAC: Adding query filter params (type=image) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:19.554172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406f354-3119-461f-bdc6-0194ef296b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:19.555540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406f354-3119-461f-bdc6-0194ef296b75 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:19.561692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 506/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:19 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:20.919997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c942a95-2f63-4ff4-be88-bbb68ac98369 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:20.920490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c942a95-2f63-4ff4-be88-bbb68ac98369 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:20.920490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c942a95-2f63-4ff4-be88-bbb68ac98369 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:20.922041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 502/1008] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:20.927288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c518ca-78d8-4b60-bc9f-c2c753d14018 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:20.928235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c518ca-78d8-4b60-bc9f-c2c753d14018 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:20.928545 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c518ca-78d8-4b60-bc9f-c2c753d14018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:20.948812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a2c518ca-78d8-4b60-bc9f-c2c753d14018 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:20.950723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a2c518ca-78d8-4b60-bc9f-c2c753d14018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:21.007088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:21.021139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 507/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:21.285016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0894bf-9ac8-4555-a489-5abf73126cc1 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:21.285373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0894bf-9ac8-4555-a489-5abf73126cc1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:21.285616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0894bf-9ac8-4555-a489-5abf73126cc1 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:21.287519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 503/1010] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:21.293123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831d4a-548c-44c4-ae0d-0fa834b63ac6 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:21.293412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831d4a-548c-44c4-ae0d-0fa834b63ac6 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:21.293687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831d4a-548c-44c4-ae0d-0fa834b63ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:21.299967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0894bf-9ac8-4555-a489-5abf73126cc1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:21.312234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8d831d4a-548c-44c4-ae0d-0fa834b63ac6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:21.313457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8d831d4a-548c-44c4-ae0d-0fa834b63ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:21.388836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 508/1011] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:21.416198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0894bf-9ac8-4555-a489-5abf73126cc1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:21.451916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c135730-d39b-44e6-bdcf-93c4a616c1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:21.452547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c135730-d39b-44e6-bdcf-93c4a616c1f2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:21.452828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c135730-d39b-44e6-bdcf-93c4a616c1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:21.453142 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c135730-d39b-44e6-bdcf-93c4a616c1f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:21.454257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c135730-d39b-44e6-bdcf-93c4a616c1f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:21.455687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c135730-d39b-44e6-bdcf-93c4a616c1f2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:21.461451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 504/1012] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:21.464137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d831d4a-548c-44c4-ae0d-0fa834b63ac6 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:21.572209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d831d4a-548c-44c4-ae0d-0fa834b63ac6 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:21.594817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:21.619056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e2816-56f4-424c-877d-00e509bcf8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:21.619790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e2816-56f4-424c-877d-00e509bcf8b8 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:21.620065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e2816-56f4-424c-877d-00e509bcf8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:21.620531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e2816-56f4-424c-877d-00e509bcf8b8 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:21.621255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e2816-56f4-424c-877d-00e509bcf8b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:21.622731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e2816-56f4-424c-877d-00e509bcf8b8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:21.659774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c135730-d39b-44e6-bdcf-93c4a616c1f2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:21.672740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:21.742936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 509/1013] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:34:21 2022] POST /identity/v3/projects => generated 311 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:34:21.743675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e617d-0c2a-4f70-90c6-2a737ecd7941 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:21.743958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e617d-0c2a-4f70-90c6-2a737ecd7941 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:21.744232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e617d-0c2a-4f70-90c6-2a737ecd7941 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:21.745474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 510/1014] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:21.751212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b7b22-22de-4f1e-8eba-876746b6f1c0 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:21.751497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b7b22-22de-4f1e-8eba-876746b6f1c0 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:21.751771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b7b22-22de-4f1e-8eba-876746b6f1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:21.770929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c135730-d39b-44e6-bdcf-93c4a616c1f2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:21.771140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-588b7b22-22de-4f1e-8eba-876746b6f1c0 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:21.772179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-588b7b22-22de-4f1e-8eba-876746b6f1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:21.811433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab90377-0451-46f6-b93d-b479474c3f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:21.812095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90377-0451-46f6-b93d-b479474c3f8c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:21.812370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90377-0451-46f6-b93d-b479474c3f8c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:21.812652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90377-0451-46f6-b93d-b479474c3f8c None admin] PATH_INFO: `/v3/services/compute` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:21.813292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab90377-0451-46f6-b93d-b479474c3f8c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:21.814728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab90377-0451-46f6-b93d-b479474c3f8c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-5ab90377-0451-46f6-b93d-b479474c3f8c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:21.824630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:21.825783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:21.826671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 505/1015] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:34:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:21.831806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab90377-0451-46f6-b93d-b479474c3f8c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:21.859786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 511/1016] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:21.917734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:21.947767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab90377-0451-46f6-b93d-b479474c3f8c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:21.984356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4ab4e-72e1-4e2b-a10d-e83a2de09633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:21.984964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4ab4e-72e1-4e2b-a10d-e83a2de09633 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:21.985282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4ab4e-72e1-4e2b-a10d-e83a2de09633 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:21.985560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4ab4e-72e1-4e2b-a10d-e83a2de09633 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:21.986197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4ab4e-72e1-4e2b-a10d-e83a2de09633 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:21.986512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4ab4e-72e1-4e2b-a10d-e83a2de09633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:21.987896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4ab4e-72e1-4e2b-a10d-e83a2de09633 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:21.992912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 506/1017] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:34:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:21.998941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b7b22-22de-4f1e-8eba-876746b6f1c0 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:22.101136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:22.104126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b7b22-22de-4f1e-8eba-876746b6f1c0 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:22.137653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4ab4e-72e1-4e2b-a10d-e83a2de09633 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:22.139985 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e668af9-0aa7-4cba-b39a-93c1d56c5c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:22.140798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e668af9-0aa7-4cba-b39a-93c1d56c5c2c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:22.141104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e668af9-0aa7-4cba-b39a-93c1d56c5c2c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:22.141387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e668af9-0aa7-4cba-b39a-93c1d56c5c2c None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:22.142023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e668af9-0aa7-4cba-b39a-93c1d56c5c2c None admin] RBAC: Adding query filter params (type=compute) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:22.142332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e668af9-0aa7-4cba-b39a-93c1d56c5c2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:22.143698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e668af9-0aa7-4cba-b39a-93c1d56c5c2c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:22.148999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 512/1018] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 8 18:34:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:22.155213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e668af9-0aa7-4cba-b39a-93c1d56c5c2c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:22.252046 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4ab4e-72e1-4e2b-a10d-e83a2de09633 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:22.265515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e668af9-0aa7-4cba-b39a-93c1d56c5c2c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:22.289456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c4e5c-114f-4ab6-8906-d569a5277eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:22.290066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c4e5c-114f-4ab6-8906-d569a5277eb1 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:22.290345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c4e5c-114f-4ab6-8906-d569a5277eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:22.290617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c4e5c-114f-4ab6-8906-d569a5277eb1 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:22.291277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c4e5c-114f-4ab6-8906-d569a5277eb1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:22.292726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c4e5c-114f-4ab6-8906-d569a5277eb1 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:22.304625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f81119-6108-4fcd-89d9-92df4b815c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:22.305370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f81119-6108-4fcd-89d9-92df4b815c27 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:22.305774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f81119-6108-4fcd-89d9-92df4b815c27 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:22.306118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f81119-6108-4fcd-89d9-92df4b815c27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:22.306860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f81119-6108-4fcd-89d9-92df4b815c27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:22.308280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f81119-6108-4fcd-89d9-92df4b815c27 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:22.398185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:22.404480 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 507/1019] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:34:22 2022] POST /identity/v3/services => generated 242 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:34:22.421539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 513/1020] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:34:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:34:24.016112 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42009de8-193c-4c89-81c8-bc85154fd069 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:24.016607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42009de8-193c-4c89-81c8-bc85154fd069 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:24.016607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42009de8-193c-4c89-81c8-bc85154fd069 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:24.018128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 508/1021] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:24.023588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d7be4-fa7e-4274-9341-8a2e38d0ff96 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:24.024015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d7be4-fa7e-4274-9341-8a2e38d0ff96 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:24.024159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d7be4-fa7e-4274-9341-8a2e38d0ff96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:24.042969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-017d7be4-fa7e-4274-9341-8a2e38d0ff96 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:24.044156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-017d7be4-fa7e-4274-9341-8a2e38d0ff96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:24.059709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:24.119223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 514/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:24.388402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42009de8-193c-4c89-81c8-bc85154fd069 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:24.498173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42009de8-193c-4c89-81c8-bc85154fd069 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:24.536567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33a3ec-1b81-44ba-94e8-6df23afd9fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:24.537400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33a3ec-1b81-44ba-94e8-6df23afd9fd3 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:24.537681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33a3ec-1b81-44ba-94e8-6df23afd9fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:24.538026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33a3ec-1b81-44ba-94e8-6df23afd9fd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:24.539324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33a3ec-1b81-44ba-94e8-6df23afd9fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:24.540797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33a3ec-1b81-44ba-94e8-6df23afd9fd3 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:24.542672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 509/1023] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:24.548578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d7be4-fa7e-4274-9341-8a2e38d0ff96 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:24.647697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a5e9a-4174-4e96-b2d8-efce1ad9b780 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:24.647998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a5e9a-4174-4e96-b2d8-efce1ad9b780 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:24.648340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a5e9a-4174-4e96-b2d8-efce1ad9b780 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:24.649785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 510/1024] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:24.654919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85fa1-7445-4dae-a3e6-a37827954086 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:24.655213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85fa1-7445-4dae-a3e6-a37827954086 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:24.655491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85fa1-7445-4dae-a3e6-a37827954086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:24.665213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d7be4-fa7e-4274-9341-8a2e38d0ff96 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:24.675628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a0d85fa1-7445-4dae-a3e6-a37827954086 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:24.677255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a0d85fa1-7445-4dae-a3e6-a37827954086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:24.700698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe411f-b4c0-45ce-92d8-21dcbc356735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:24.701352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe411f-b4c0-45ce-92d8-21dcbc356735 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:24.701626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe411f-b4c0-45ce-92d8-21dcbc356735 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:24.701909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe411f-b4c0-45ce-92d8-21dcbc356735 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:24.702571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe411f-b4c0-45ce-92d8-21dcbc356735 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:24.703993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe411f-b4c0-45ce-92d8-21dcbc356735 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:24.739659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:24.747599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 511/1025] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:24.748952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f940a89-5afb-422e-9497-63292d50ea76 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:24.749396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f940a89-5afb-422e-9497-63292d50ea76 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:24.749676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f940a89-5afb-422e-9497-63292d50ea76 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:24.750921 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 512/1026] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:24.756229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf87e1-1d33-4ffc-8447-7b529bd95531 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:24.756517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf87e1-1d33-4ffc-8447-7b529bd95531 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:24.756795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf87e1-1d33-4ffc-8447-7b529bd95531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:24.774150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-4baf87e1-1d33-4ffc-8447-7b529bd95531 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:24.775338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4baf87e1-1d33-4ffc-8447-7b529bd95531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:24.826263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 515/1027] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:34:24 2022] POST /identity/v3/users => generated 285 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:34:24.843074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 513/1028] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:25.025230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe411f-b4c0-45ce-92d8-21dcbc356735 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:25.028262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:25.125184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf87e1-1d33-4ffc-8447-7b529bd95531 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:25.143677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe411f-b4c0-45ce-92d8-21dcbc356735 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:25.181695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011793a4-5c1c-4428-b00b-c2d82216da57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:25.182339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011793a4-5c1c-4428-b00b-c2d82216da57 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:25.182613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011793a4-5c1c-4428-b00b-c2d82216da57 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:25.182887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011793a4-5c1c-4428-b00b-c2d82216da57 None admin] PATH_INFO: `/v3/services/image` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:25.183467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011793a4-5c1c-4428-b00b-c2d82216da57 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:25.185344 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011793a4-5c1c-4428-b00b-c2d82216da57 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-011793a4-5c1c-4428-b00b-c2d82216da57 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:25.195104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:25.196136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:25.197008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 516/1029] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 8 18:34:25 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:25.202767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011793a4-5c1c-4428-b00b-c2d82216da57 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:25.242936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf87e1-1d33-4ffc-8447-7b529bd95531 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:25.271890 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:25.275595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:25.281600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3665a0-6445-496b-a2b5-55b627894b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:25.283117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3665a0-6445-496b-a2b5-55b627894b09 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:25.283117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3665a0-6445-496b-a2b5-55b627894b09 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:25.283117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3665a0-6445-496b-a2b5-55b627894b09 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:25.293681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3665a0-6445-496b-a2b5-55b627894b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-1a3665a0-6445-496b-a2b5-55b627894b09 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:34:25.298405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:25.299340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:25.300225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 514/1030] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 18:34:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:25.305569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3665a0-6445-496b-a2b5-55b627894b09 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:25.315849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011793a4-5c1c-4428-b00b-c2d82216da57 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:25.355127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76b92-3c69-4381-b9fe-3a0d5c607c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:25.355739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76b92-3c69-4381-b9fe-3a0d5c607c0b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:25.356086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76b92-3c69-4381-b9fe-3a0d5c607c0b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:25.356335 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76b92-3c69-4381-b9fe-3a0d5c607c0b None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:25.356981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc76b92-3c69-4381-b9fe-3a0d5c607c0b None admin] RBAC: Adding query filter params (name=image) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:25.357305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc76b92-3c69-4381-b9fe-3a0d5c607c0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:25.358660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc76b92-3c69-4381-b9fe-3a0d5c607c0b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:25.363948 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 517/1031] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:25.370487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76b92-3c69-4381-b9fe-3a0d5c607c0b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:25.422042 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3665a0-6445-496b-a2b5-55b627894b09 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:25.458233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fbf08-942d-4b2d-891c-7b234430eb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:25.458888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fbf08-942d-4b2d-891c-7b234430eb46 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:25.459176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fbf08-942d-4b2d-891c-7b234430eb46 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:25.459447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fbf08-942d-4b2d-891c-7b234430eb46 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:25.460117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fbf08-942d-4b2d-891c-7b234430eb46 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:25.460423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fbf08-942d-4b2d-891c-7b234430eb46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:25.461970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fbf08-942d-4b2d-891c-7b234430eb46 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:25.470662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 515/1032] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 18:34:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:25.477471 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fbf08-942d-4b2d-891c-7b234430eb46 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:25.484454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76b92-3c69-4381-b9fe-3a0d5c607c0b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:25.518696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:25.525337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79823e4-284e-4e0f-9c05-a949186ef5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:25.525995 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79823e4-284e-4e0f-9c05-a949186ef5ce None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:25.526216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79823e4-284e-4e0f-9c05-a949186ef5ce None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:25.526480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79823e4-284e-4e0f-9c05-a949186ef5ce None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:25.527109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79823e4-284e-4e0f-9c05-a949186ef5ce None admin] RBAC: Adding query filter params (type=image) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:25.527410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79823e4-284e-4e0f-9c05-a949186ef5ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:25.528236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:25.528753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79823e4-284e-4e0f-9c05-a949186ef5ce None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:25.534081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 518/1033] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:25.540397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79823e4-284e-4e0f-9c05-a949186ef5ce None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:25.592908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fbf08-942d-4b2d-891c-7b234430eb46 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:25.633909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4acb2e-5df5-4311-8597-963fc2e9dac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:25.634583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4acb2e-5df5-4311-8597-963fc2e9dac4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:25.634899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4acb2e-5df5-4311-8597-963fc2e9dac4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:25.635174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4acb2e-5df5-4311-8597-963fc2e9dac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:25.647574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4acb2e-5df5-4311-8597-963fc2e9dac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:25.649113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4acb2e-5df5-4311-8597-963fc2e9dac4 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:25.654777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79823e4-284e-4e0f-9c05-a949186ef5ce None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-3c4acb2e-5df5-4311-8597-963fc2e9dac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:25.663740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:25.664960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:25.666085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:25.666085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:25.666085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:25.666085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 516/1034] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:25.670531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4acb2e-5df5-4311-8597-963fc2e9dac4 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:25.698070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adccae-2bdc-4a56-8627-1b28200aed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:25.698678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adccae-2bdc-4a56-8627-1b28200aed46 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:25.698958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adccae-2bdc-4a56-8627-1b28200aed46 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:25.699229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adccae-2bdc-4a56-8627-1b28200aed46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:25.699922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adccae-2bdc-4a56-8627-1b28200aed46 None admin] RBAC: Adding query filter params (interface=public, service_id=066a9f99e15642df833d0062ff30fbe9, region_id=RegionOne) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:25.700231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adccae-2bdc-4a56-8627-1b28200aed46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:25.701626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adccae-2bdc-4a56-8627-1b28200aed46 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:25.711674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 519/1035] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 18:34:25 2022] GET /identity/v3/endpoints?service_id=066a9f99e15642df833d0062ff30fbe9&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:25.713017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adccae-2bdc-4a56-8627-1b28200aed46 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:25.774599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:25.776754 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:25.780501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4acb2e-5df5-4311-8597-963fc2e9dac4 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:25.817692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190b01a-8806-4e57-8fa1-b24d38a1e633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:25.818841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b01a-8806-4e57-8fa1-b24d38a1e633 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:25.819115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b01a-8806-4e57-8fa1-b24d38a1e633 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:25.819534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b01a-8806-4e57-8fa1-b24d38a1e633 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:25.820167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190b01a-8806-4e57-8fa1-b24d38a1e633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:25.820462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190b01a-8806-4e57-8fa1-b24d38a1e633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:25.821911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190b01a-8806-4e57-8fa1-b24d38a1e633 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:25.822684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adccae-2bdc-4a56-8627-1b28200aed46 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:25.834277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 517/1036] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:25.840038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190b01a-8806-4e57-8fa1-b24d38a1e633 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:25.864316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d436e-9c11-409c-b7f3-13ee818c2f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:25.864926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d436e-9c11-409c-b7f3-13ee818c2f1b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:25.865783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d436e-9c11-409c-b7f3-13ee818c2f1b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:25.866059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d436e-9c11-409c-b7f3-13ee818c2f1b None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:25.866751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d436e-9c11-409c-b7f3-13ee818c2f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:25.868107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d436e-9c11-409c-b7f3-13ee818c2f1b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:25.874591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 520/1037] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 18:34:25 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:34:25.957348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190b01a-8806-4e57-8fa1-b24d38a1e633 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:25.999890 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d8f4c-a1df-448c-8a3b-17e01107eba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:26.000860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d8f4c-a1df-448c-8a3b-17e01107eba7 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:26.001373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d8f4c-a1df-448c-8a3b-17e01107eba7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:26.001826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d8f4c-a1df-448c-8a3b-17e01107eba7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:26.011879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d8f4c-a1df-448c-8a3b-17e01107eba7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:26.013786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d8f4c-a1df-448c-8a3b-17e01107eba7 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-4e1d8f4c-a1df-448c-8a3b-17e01107eba7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:34:26.026732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:26.027809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:26.028760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:26.028760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:26.028760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:34:26.028760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:34:26.028760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:34:26.028760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:34:26.028760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:34:26.028760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:34:26.028760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:34:26.028760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:26.030617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 518/1038] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 18:34:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:26.036518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d436e-9c11-409c-b7f3-13ee818c2f1b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:26.141599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d436e-9c11-409c-b7f3-13ee818c2f1b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:26.166398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:26.178340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b71f7-7064-4276-a9b7-ba1beb9b71f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:26.179059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b71f7-7064-4276-a9b7-ba1beb9b71f8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:26.179341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b71f7-7064-4276-a9b7-ba1beb9b71f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:26.179616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b71f7-7064-4276-a9b7-ba1beb9b71f8 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:26.180346 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b71f7-7064-4276-a9b7-ba1beb9b71f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:26.183771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b71f7-7064-4276-a9b7-ba1beb9b71f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:26.183771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b71f7-7064-4276-a9b7-ba1beb9b71f8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:26.201720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 521/1039] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 18:34:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:26.207331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d8f4c-a1df-448c-8a3b-17e01107eba7 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:26.232183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:26.312584 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d8f4c-a1df-448c-8a3b-17e01107eba7 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:26.347776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec4db9-b72c-4cc1-9590-4a92a4e514a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:26.348386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec4db9-b72c-4cc1-9590-4a92a4e514a9 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:26.348668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec4db9-b72c-4cc1-9590-4a92a4e514a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:26.348945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec4db9-b72c-4cc1-9590-4a92a4e514a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:26.419144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec4db9-b72c-4cc1-9590-4a92a4e514a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:26.420969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec4db9-b72c-4cc1-9590-4a92a4e514a9 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:26.442410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:26.532650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 519/1040] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:34:26 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 18:34:26.534118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b71f7-7064-4276-a9b7-ba1beb9b71f8 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:26.642475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b71f7-7064-4276-a9b7-ba1beb9b71f8 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:26.679007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9aea1-317f-41db-a121-594e481dfe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:26.679635 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9aea1-317f-41db-a121-594e481dfe95 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:26.679910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9aea1-317f-41db-a121-594e481dfe95 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:26.680176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9aea1-317f-41db-a121-594e481dfe95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:26.690796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9aea1-317f-41db-a121-594e481dfe95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:26.692235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9aea1-317f-41db-a121-594e481dfe95 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-b4a9aea1-317f-41db-a121-594e481dfe95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:26.707441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:26.708521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:26.710069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:26.710069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:26.710069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:26.710069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 522/1041] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:26.715068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec4db9-b72c-4cc1-9590-4a92a4e514a9 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:26.819234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec4db9-b72c-4cc1-9590-4a92a4e514a9 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:26.839342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:26.853788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5777c-ad33-45ad-a99f-f0c194fc21cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:26.864481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5777c-ad33-45ad-a99f-f0c194fc21cc None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:26.864481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5777c-ad33-45ad-a99f-f0c194fc21cc None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:26.864481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5777c-ad33-45ad-a99f-f0c194fc21cc None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:26.864481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5777c-ad33-45ad-a99f-f0c194fc21cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:26.864481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5777c-ad33-45ad-a99f-f0c194fc21cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:26.864481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5777c-ad33-45ad-a99f-f0c194fc21cc None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:26.873966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 520/1042] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:26.875260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9aea1-317f-41db-a121-594e481dfe95 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:26.909297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:26.984236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9aea1-317f-41db-a121-594e481dfe95 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:27.018890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe321e5c-2075-4690-8406-ed93a1c04b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:27.019519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe321e5c-2075-4690-8406-ed93a1c04b90 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:27.019791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe321e5c-2075-4690-8406-ed93a1c04b90 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:27.020060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe321e5c-2075-4690-8406-ed93a1c04b90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:27.027473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe321e5c-2075-4690-8406-ed93a1c04b90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:27.028900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe321e5c-2075-4690-8406-ed93a1c04b90 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-fe321e5c-2075-4690-8406-ed93a1c04b90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:34:27.040592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:27.042103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:27.043453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:34:27.043453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:34:27.043453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:34:27.043453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:34:27.043453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:34:27.043453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:34:27.043453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:27.043453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:27.045128 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 523/1043] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:27.047611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5777c-ad33-45ad-a99f-f0c194fc21cc None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:27.065563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e6804-a02c-438a-bc12-1f88e76138f1 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:27.065852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e6804-a02c-438a-bc12-1f88e76138f1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:27.066110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e6804-a02c-438a-bc12-1f88e76138f1 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:27.067566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 524/1044] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:34:27.072509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b917-916b-49c7-96e3-cbc68d9f1c14 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:27.072792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b917-916b-49c7-96e3-cbc68d9f1c14 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:27.073088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7b917-916b-49c7-96e3-cbc68d9f1c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:27.091429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d2d7b917-916b-49c7-96e3-cbc68d9f1c14 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:27.092616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d2d7b917-916b-49c7-96e3-cbc68d9f1c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:27.160305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5777c-ad33-45ad-a99f-f0c194fc21cc None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:27.169227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 525/1045] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:27.195969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83bf12-35c6-4968-861a-5ce82165e4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:27.196581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83bf12-35c6-4968-861a-5ce82165e4cd None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:27.196859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83bf12-35c6-4968-861a-5ce82165e4cd None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:27.197160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83bf12-35c6-4968-861a-5ce82165e4cd None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:27.197849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83bf12-35c6-4968-861a-5ce82165e4cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:27.198154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83bf12-35c6-4968-861a-5ce82165e4cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:27.199569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83bf12-35c6-4968-861a-5ce82165e4cd None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:27.216236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 521/1046] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:27.218220 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7b917-916b-49c7-96e3-cbc68d9f1c14 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:27.243136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:27.319168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7b917-916b-49c7-96e3-cbc68d9f1c14 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:27.352892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fda708-17a8-4eca-bd05-317184a66bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:27.353517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fda708-17a8-4eca-bd05-317184a66bae None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:27.353796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fda708-17a8-4eca-bd05-317184a66bae None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:27.354071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fda708-17a8-4eca-bd05-317184a66bae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:27.354781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fda708-17a8-4eca-bd05-317184a66bae None admin] RBAC: Adding query filter params (role.id=8a81cd8421c8438cb3a40fd7e56a97eb, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=1b29c5745a8e42ff8232e5ca22b4a4ad) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:27.355086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fda708-17a8-4eca-bd05-317184a66bae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:27.356481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fda708-17a8-4eca-bd05-317184a66bae None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:27.361833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 526/1047] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:34:27 2022] GET /identity/v3/role_assignments?user.id=1b29c5745a8e42ff8232e5ca22b4a4ad&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=8a81cd8421c8438cb3a40fd7e56a97eb => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:27.466730 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83bf12-35c6-4968-861a-5ce82165e4cd None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:27.513966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:27.576766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83bf12-35c6-4968-861a-5ce82165e4cd None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:27.616132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54212ca4-975e-4b01-a964-0cfb1e2c9120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:27.616766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54212ca4-975e-4b01-a964-0cfb1e2c9120 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:27.617133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54212ca4-975e-4b01-a964-0cfb1e2c9120 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:27.617417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54212ca4-975e-4b01-a964-0cfb1e2c9120 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:27.629514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54212ca4-975e-4b01-a964-0cfb1e2c9120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-54212ca4-975e-4b01-a964-0cfb1e2c9120 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:34:27.634863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:27.635821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:27.636678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 522/1048] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:34:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:27.643633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fda708-17a8-4eca-bd05-317184a66bae None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:27.738992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:27.754506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fda708-17a8-4eca-bd05-317184a66bae None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:27.789604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea668a8-4f33-4bdd-a0b8-5a62e328fb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:27.790215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea668a8-4f33-4bdd-a0b8-5a62e328fb77 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:27.790491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea668a8-4f33-4bdd-a0b8-5a62e328fb77 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:27.790761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea668a8-4f33-4bdd-a0b8-5a62e328fb77 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:27.791425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea668a8-4f33-4bdd-a0b8-5a62e328fb77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:27.791725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea668a8-4f33-4bdd-a0b8-5a62e328fb77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:27.793129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea668a8-4f33-4bdd-a0b8-5a62e328fb77 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:27.806836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 527/1049] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:34:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:27.808186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54212ca4-975e-4b01-a964-0cfb1e2c9120 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:27.910291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:27.914040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54212ca4-975e-4b01-a964-0cfb1e2c9120 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:27.953770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e570b-ab68-4d1b-9867-1efd2bb0e44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:27.954859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e570b-ab68-4d1b-9867-1efd2bb0e44d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:27.955349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e570b-ab68-4d1b-9867-1efd2bb0e44d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:27.955827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e570b-ab68-4d1b-9867-1efd2bb0e44d None admin] PATH_INFO: `/v3/users/4daa6ad53cc64ed58c7b8bf7cab56996` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:27.973229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e570b-ab68-4d1b-9867-1efd2bb0e44d None admin] RBAC: Authorizing `identity:get_user(user_id=4daa6ad53cc64ed58c7b8bf7cab56996)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:27.974814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e570b-ab68-4d1b-9867-1efd2bb0e44d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:27.976980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 523/1050] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:34:27 2022] GET /identity/v3/users/4daa6ad53cc64ed58c7b8bf7cab56996 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:27.982711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea668a8-4f33-4bdd-a0b8-5a62e328fb77 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:28.078412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbbc7e-0d48-4512-910f-71fbc5bef567 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:28.078774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbbc7e-0d48-4512-910f-71fbc5bef567 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:28.078976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbbc7e-0d48-4512-910f-71fbc5bef567 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:28.080329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 524/1051] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:28.086205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479eebd-9f5c-4514-8002-2f46fc81b57f None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:28.086601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479eebd-9f5c-4514-8002-2f46fc81b57f None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:28.086876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479eebd-9f5c-4514-8002-2f46fc81b57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:28.090065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea668a8-4f33-4bdd-a0b8-5a62e328fb77 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:28.105868 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b479eebd-9f5c-4514-8002-2f46fc81b57f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:28.107060 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b479eebd-9f5c-4514-8002-2f46fc81b57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:28.127122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e9d3a-4041-4920-9e2a-f337bcddcb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:28.127729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e9d3a-4041-4920-9e2a-f337bcddcb12 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:28.128010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e9d3a-4041-4920-9e2a-f337bcddcb12 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:28.128282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e9d3a-4041-4920-9e2a-f337bcddcb12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:28.148437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:28.184186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 525/1052] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:28.201877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e9d3a-4041-4920-9e2a-f337bcddcb12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:28.203668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e9d3a-4041-4920-9e2a-f337bcddcb12 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:28.315437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 528/1053] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:34:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:34:28.320972 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479eebd-9f5c-4514-8002-2f46fc81b57f None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:28.408449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:28.421218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479eebd-9f5c-4514-8002-2f46fc81b57f None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:28.455772 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c40d3-36f8-4c2f-9c89-7bbbfae328d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:28.456392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c40d3-36f8-4c2f-9c89-7bbbfae328d9 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:28.456671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c40d3-36f8-4c2f-9c89-7bbbfae328d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:28.456940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c40d3-36f8-4c2f-9c89-7bbbfae328d9 None admin] PATH_INFO: `/v3/projects/c172446498ba49dc832907a9acccefa7` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:28.457423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e9d3a-4041-4920-9e2a-f337bcddcb12 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:28.458259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c40d3-36f8-4c2f-9c89-7bbbfae328d9 None admin] RBAC: Authorizing `identity:get_project(project_id=c172446498ba49dc832907a9acccefa7)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:28.459814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c40d3-36f8-4c2f-9c89-7bbbfae328d9 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:28.465361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 526/1054] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:34:28 2022] GET /identity/v3/projects/c172446498ba49dc832907a9acccefa7 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:28.468394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c40d3-36f8-4c2f-9c89-7bbbfae328d9 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:28.517263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:28.576945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e9d3a-4041-4920-9e2a-f337bcddcb12 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:28.578336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c40d3-36f8-4c2f-9c89-7bbbfae328d9 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:28.627405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40630f31-cea7-478d-9eaa-67d14e0da2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:28.627760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272902ee-cfc4-469f-8593-0d2b02ff6d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:28.628401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630f31-cea7-478d-9eaa-67d14e0da2e7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:28.628705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272902ee-cfc4-469f-8593-0d2b02ff6d2c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:28.628879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630f31-cea7-478d-9eaa-67d14e0da2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:28.629194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272902ee-cfc4-469f-8593-0d2b02ff6d2c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:28.629308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40630f31-cea7-478d-9eaa-67d14e0da2e7 None admin] PATH_INFO: `/v3/services/image` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:28.629639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272902ee-cfc4-469f-8593-0d2b02ff6d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:28.630343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40630f31-cea7-478d-9eaa-67d14e0da2e7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:28.631408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272902ee-cfc4-469f-8593-0d2b02ff6d2c None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=c172446498ba49dc832907a9acccefa7, user.id=4daa6ad53cc64ed58c7b8bf7cab56996) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:28.631978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272902ee-cfc4-469f-8593-0d2b02ff6d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:28.634112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40630f31-cea7-478d-9eaa-67d14e0da2e7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:28.634333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272902ee-cfc4-469f-8593-0d2b02ff6d2c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:28.653297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 527/1055] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:34:28 2022] GET /identity/v3/role_assignments?user.id=4daa6ad53cc64ed58c7b8bf7cab56996&scope.project.id=c172446498ba49dc832907a9acccefa7&role.id=5c777de40b8449329d714d8c2229115e => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-40630f31-cea7-478d-9eaa-67d14e0da2e7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:28.653509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:28.654683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:28.655798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 529/1056] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 8 18:34:28 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:28.657012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272902ee-cfc4-469f-8593-0d2b02ff6d2c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:28.684226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:28.779312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272902ee-cfc4-469f-8593-0d2b02ff6d2c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:28.817866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc620ed-ca3f-4118-9aa3-ce9f8ac59ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:28.819896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc620ed-ca3f-4118-9aa3-ce9f8ac59ca2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:28.820185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc620ed-ca3f-4118-9aa3-ce9f8ac59ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:28.820450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc620ed-ca3f-4118-9aa3-ce9f8ac59ca2 None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:28.821110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc620ed-ca3f-4118-9aa3-ce9f8ac59ca2 None admin] RBAC: Adding query filter params (name=image) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:28.821435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc620ed-ca3f-4118-9aa3-ce9f8ac59ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:28.822786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc620ed-ca3f-4118-9aa3-ce9f8ac59ca2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:28.828445 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 528/1057] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:28.835421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40630f31-cea7-478d-9eaa-67d14e0da2e7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:28.945492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40630f31-cea7-478d-9eaa-67d14e0da2e7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:28.960817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:28.984222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ce8b1-0497-4def-a789-8ae9d1d5747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:28.985144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ce8b1-0497-4def-a789-8ae9d1d5747f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:28.985592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ce8b1-0497-4def-a789-8ae9d1d5747f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:28.986013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ce8b1-0497-4def-a789-8ae9d1d5747f None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:28.986784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ce8b1-0497-4def-a789-8ae9d1d5747f None admin] RBAC: Adding query filter params (type=image) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:28.987234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ce8b1-0497-4def-a789-8ae9d1d5747f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:28.988748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ce8b1-0497-4def-a789-8ae9d1d5747f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:28.999146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 530/1058] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:29.000913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc620ed-ca3f-4118-9aa3-ce9f8ac59ca2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:29.090404 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:29.107566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc620ed-ca3f-4118-9aa3-ce9f8ac59ca2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:29.145431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3dcce-08f4-4cdf-bffe-6360096e0549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:29.146045 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3dcce-08f4-4cdf-bffe-6360096e0549 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:29.146325 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3dcce-08f4-4cdf-bffe-6360096e0549 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:29.147078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3dcce-08f4-4cdf-bffe-6360096e0549 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:29.147758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3dcce-08f4-4cdf-bffe-6360096e0549 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:29.149135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3dcce-08f4-4cdf-bffe-6360096e0549 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:29.273162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 529/1059] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:34:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:34:29.571286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512960ff-5d56-46a1-b30f-e6a004cd6663 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:29.571824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512960ff-5d56-46a1-b30f-e6a004cd6663 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:29.571824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512960ff-5d56-46a1-b30f-e6a004cd6663 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:29.573517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 531/1060] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:29.578470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465a87d-b3f3-49e3-af53-ecdfa3cbd51d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:29.578850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465a87d-b3f3-49e3-af53-ecdfa3cbd51d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:29.579029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465a87d-b3f3-49e3-af53-ecdfa3cbd51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:29.598228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2465a87d-b3f3-49e3-af53-ecdfa3cbd51d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:29.599599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2465a87d-b3f3-49e3-af53-ecdfa3cbd51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:29.676416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 530/1061] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:29.946845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512960ff-5d56-46a1-b30f-e6a004cd6663 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:30.053550 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512960ff-5d56-46a1-b30f-e6a004cd6663 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:30.088414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3e547-0c32-4912-a76a-ddea76905adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:30.089066 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3e547-0c32-4912-a76a-ddea76905adc None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:30.089359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3e547-0c32-4912-a76a-ddea76905adc None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:30.089634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3e547-0c32-4912-a76a-ddea76905adc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:30.100899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3e547-0c32-4912-a76a-ddea76905adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-57d3e547-0c32-4912-a76a-ddea76905adc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:34:30.105779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:30.106698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:30.107563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:30.108991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 532/1062] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 18:34:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:30.112867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465a87d-b3f3-49e3-af53-ecdfa3cbd51d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:30.179180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:30.219109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465a87d-b3f3-49e3-af53-ecdfa3cbd51d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:30.255307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6c194-b1d0-4429-9cb3-90ef67c0f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:30.256572 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6c194-b1d0-4429-9cb3-90ef67c0f7a6 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:30.256858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6c194-b1d0-4429-9cb3-90ef67c0f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:30.257165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6c194-b1d0-4429-9cb3-90ef67c0f7a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:30.257877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6c194-b1d0-4429-9cb3-90ef67c0f7a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:30.258189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6c194-b1d0-4429-9cb3-90ef67c0f7a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:30.259555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6c194-b1d0-4429-9cb3-90ef67c0f7a6 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:30.272988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 531/1063] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 18:34:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:30.274606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3e547-0c32-4912-a76a-ddea76905adc None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:30.310429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:30.378469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3e547-0c32-4912-a76a-ddea76905adc None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:30.415489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bf5f9-1c2d-48cf-a0b9-4ac56dc40bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:30.416180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bf5f9-1c2d-48cf-a0b9-4ac56dc40bc5 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:30.416463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bf5f9-1c2d-48cf-a0b9-4ac56dc40bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:30.416736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bf5f9-1c2d-48cf-a0b9-4ac56dc40bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:30.427121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367bf5f9-1c2d-48cf-a0b9-4ac56dc40bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:30.429138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367bf5f9-1c2d-48cf-a0b9-4ac56dc40bc5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-367bf5f9-1c2d-48cf-a0b9-4ac56dc40bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:30.444265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:30.445278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:30.450936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:30.450936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:30.450936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:30.450936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 533/1064] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:30.452665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6c194-b1d0-4429-9cb3-90ef67c0f7a6 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:30.560222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6c194-b1d0-4429-9cb3-90ef67c0f7a6 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:30.587806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:30.599087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57cde5-de02-46d2-9bd7-610722227400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:30.600453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57cde5-de02-46d2-9bd7-610722227400 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:30.605748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57cde5-de02-46d2-9bd7-610722227400 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:30.605748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57cde5-de02-46d2-9bd7-610722227400 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:30.605748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57cde5-de02-46d2-9bd7-610722227400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:30.605748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57cde5-de02-46d2-9bd7-610722227400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:30.605748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57cde5-de02-46d2-9bd7-610722227400 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:30.621997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 532/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:30.623756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bf5f9-1c2d-48cf-a0b9-4ac56dc40bc5 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:30.719471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:30.732912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bf5f9-1c2d-48cf-a0b9-4ac56dc40bc5 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:30.768095 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e7537-457e-4f34-a6cc-f5c2a1fa72e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:30.768726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e7537-457e-4f34-a6cc-f5c2a1fa72e0 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:30.769115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e7537-457e-4f34-a6cc-f5c2a1fa72e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:30.769399 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e7537-457e-4f34-a6cc-f5c2a1fa72e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:30.777130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e7537-457e-4f34-a6cc-f5c2a1fa72e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:30.778569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e7537-457e-4f34-a6cc-f5c2a1fa72e0 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-815e7537-457e-4f34-a6cc-f5c2a1fa72e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:34:30.790487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:30.791441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:30.797313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 534/1066] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 18:34:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:30.799581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57cde5-de02-46d2-9bd7-610722227400 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:30.803455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de977199-581a-4838-ab39-33d5b000f399 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:30.803739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de977199-581a-4838-ab39-33d5b000f399 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:30.804017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de977199-581a-4838-ab39-33d5b000f399 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:30.805367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 535/1067] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:34:30.810560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1716f-6d7e-48df-92b5-aba4a0c529da None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:30.811004 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1716f-6d7e-48df-92b5-aba4a0c529da None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:30.811454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1716f-6d7e-48df-92b5-aba4a0c529da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:30.830850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-22e1716f-6d7e-48df-92b5-aba4a0c529da None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:30.832720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-22e1716f-6d7e-48df-92b5-aba4a0c529da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:30.906660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 536/1068] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:34:30.909838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57cde5-de02-46d2-9bd7-610722227400 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:30.946308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff505a5-b167-4e7e-a453-0f15b45849b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:30.946926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff505a5-b167-4e7e-a453-0f15b45849b8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:30.947206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff505a5-b167-4e7e-a453-0f15b45849b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:30.948703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff505a5-b167-4e7e-a453-0f15b45849b8 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:30.948703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff505a5-b167-4e7e-a453-0f15b45849b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:30.948703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff505a5-b167-4e7e-a453-0f15b45849b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:30.949966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff505a5-b167-4e7e-a453-0f15b45849b8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:30.968224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 533/1069] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 18:34:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:30.974205 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1716f-6d7e-48df-92b5-aba4a0c529da None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:31.054593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:31.077455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1716f-6d7e-48df-92b5-aba4a0c529da None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:31.111944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a051f-54d9-478f-93c2-297762c6cef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:31.112581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a051f-54d9-478f-93c2-297762c6cef2 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.112858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a051f-54d9-478f-93c2-297762c6cef2 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.113162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a051f-54d9-478f-93c2-297762c6cef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.123923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a051f-54d9-478f-93c2-297762c6cef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:31.125384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a051f-54d9-478f-93c2-297762c6cef2 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-673a051f-54d9-478f-93c2-297762c6cef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:31.140182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:31.141482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:31.142436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:31.142436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:31.142436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:31.145362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 537/1070] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:31.147841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff505a5-b167-4e7e-a453-0f15b45849b8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:31.179965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a051f-54d9-478f-93c2-297762c6cef2 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:31.199227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:31.259180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff505a5-b167-4e7e-a453-0f15b45849b8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:31.299407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a051f-54d9-478f-93c2-297762c6cef2 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:31.300750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f3fa-7259-4d7c-9868-5b5c41d695d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:31.301393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f3fa-7259-4d7c-9868-5b5c41d695d8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.301666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f3fa-7259-4d7c-9868-5b5c41d695d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.301937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f3fa-7259-4d7c-9868-5b5c41d695d8 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.302722 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205f3fa-7259-4d7c-9868-5b5c41d695d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:31.303115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205f3fa-7259-4d7c-9868-5b5c41d695d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:31.304506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205f3fa-7259-4d7c-9868-5b5c41d695d8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:31.321514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 534/1071] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:31.322720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f3fa-7259-4d7c-9868-5b5c41d695d8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:31.337484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:31.344146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3a635-f706-4d55-a7cb-733baf697277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:31.344804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3a635-f706-4d55-a7cb-733baf697277 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.345382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3a635-f706-4d55-a7cb-733baf697277 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.345657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3a635-f706-4d55-a7cb-733baf697277 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.358498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3a635-f706-4d55-a7cb-733baf697277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-02c3a635-f706-4d55-a7cb-733baf697277 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:34:31.364654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:31.369168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:31.370100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 538/1072] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:31.372463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3a635-f706-4d55-a7cb-733baf697277 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:31.438996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f3fa-7259-4d7c-9868-5b5c41d695d8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:31.447291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:31.475224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401b43c-81b4-4998-ba18-1e5497cc35a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:31.475923 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b43c-81b4-4998-ba18-1e5497cc35a6 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.476227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b43c-81b4-4998-ba18-1e5497cc35a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.476557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b43c-81b4-4998-ba18-1e5497cc35a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.483421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401b43c-81b4-4998-ba18-1e5497cc35a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:31.484947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401b43c-81b4-4998-ba18-1e5497cc35a6 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:31.490789 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3a635-f706-4d55-a7cb-733baf697277 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-3401b43c-81b4-4998-ba18-1e5497cc35a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:34:31.497379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:31.498303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:31.499336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:34:31.499336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:34:31.499336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:34:31.499336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:34:31.499336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:34:31.499336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:34:31.499336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:31.499336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:31.499336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 535/1073] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:31.504146 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401b43c-81b4-4998-ba18-1e5497cc35a6 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:31.527964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d075f75-699f-429f-b107-4f8d8343909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:31.528572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d075f75-699f-429f-b107-4f8d8343909c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.528848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d075f75-699f-429f-b107-4f8d8343909c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.529139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d075f75-699f-429f-b107-4f8d8343909c None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.529823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d075f75-699f-429f-b107-4f8d8343909c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:31.530122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d075f75-699f-429f-b107-4f8d8343909c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:31.531463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d075f75-699f-429f-b107-4f8d8343909c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:31.540069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 539/1074] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:34:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:31.541249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df404fa-1d06-4663-b755-054a81d0be71 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.541526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df404fa-1d06-4663-b755-054a81d0be71 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.541798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df404fa-1d06-4663-b755-054a81d0be71 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.542979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 540/1075] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:31.545810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df404fa-1d06-4663-b755-054a81d0be71 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:31.607320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:31.628959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401b43c-81b4-4998-ba18-1e5497cc35a6 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:31.655279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df404fa-1d06-4663-b755-054a81d0be71 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:31.664731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26e2f6-bcdd-48dd-bb81-d88a02240c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:31.665395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26e2f6-bcdd-48dd-bb81-d88a02240c86 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.665676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26e2f6-bcdd-48dd-bb81-d88a02240c86 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.665946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26e2f6-bcdd-48dd-bb81-d88a02240c86 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.666630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26e2f6-bcdd-48dd-bb81-d88a02240c86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:31.666936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26e2f6-bcdd-48dd-bb81-d88a02240c86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:31.668366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26e2f6-bcdd-48dd-bb81-d88a02240c86 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:31.681476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 536/1076] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:31.682646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cc143-3d81-491c-8a41-b3b6a99bd977 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.682929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cc143-3d81-491c-8a41-b3b6a99bd977 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.683201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cc143-3d81-491c-8a41-b3b6a99bd977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.697267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb56609-7fa6-4d78-9891-07d71034324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:31.697905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb56609-7fa6-4d78-9891-07d71034324d None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.698188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb56609-7fa6-4d78-9891-07d71034324d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.698466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb56609-7fa6-4d78-9891-07d71034324d None admin] PATH_INFO: `/v3/users/4daa6ad53cc64ed58c7b8bf7cab56996` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.699609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb56609-7fa6-4d78-9891-07d71034324d None admin] RBAC: Authorizing `identity:get_user(user_id=4daa6ad53cc64ed58c7b8bf7cab56996)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:31.700653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-394cc143-3d81-491c-8a41-b3b6a99bd977 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:31.701027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb56609-7fa6-4d78-9891-07d71034324d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:31.701864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-394cc143-3d81-491c-8a41-b3b6a99bd977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:31.707303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 541/1077] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:34:31 2022] GET /identity/v3/users/4daa6ad53cc64ed58c7b8bf7cab56996 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:31.707303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb56609-7fa6-4d78-9891-07d71034324d None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:31.722814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:31.768978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 537/1078] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:31.769676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cc143-3d81-491c-8a41-b3b6a99bd977 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:31.803347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb56609-7fa6-4d78-9891-07d71034324d None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:31.827010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:31.842275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031a79a-b98c-4bf5-83e5-71a6ed75c4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:31.842936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031a79a-b98c-4bf5-83e5-71a6ed75c4df None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.843209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031a79a-b98c-4bf5-83e5-71a6ed75c4df None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.843477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031a79a-b98c-4bf5-83e5-71a6ed75c4df None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/1b29c5745a8e42ff8232e5ca22b4a4ad/roles/8a81cd8421c8438cb3a40fd7e56a97eb` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.845916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031a79a-b98c-4bf5-83e5-71a6ed75c4df None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=1b29c5745a8e42ff8232e5ca22b4a4ad, role_id=8a81cd8421c8438cb3a40fd7e56a97eb)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:31.847336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031a79a-b98c-4bf5-83e5-71a6ed75c4df None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:31.875942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cc143-3d81-491c-8a41-b3b6a99bd977 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:31.912971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a015a-912a-4055-bec3-ed88f91341b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:31.913723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a015a-912a-4055-bec3-ed88f91341b2 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:31.914076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a015a-912a-4055-bec3-ed88f91341b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:31.914418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a015a-912a-4055-bec3-ed88f91341b2 None admin] PATH_INFO: `/v3/projects/c172446498ba49dc832907a9acccefa7` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:31.915612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a015a-912a-4055-bec3-ed88f91341b2 None admin] RBAC: Authorizing `identity:get_project(project_id=c172446498ba49dc832907a9acccefa7)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:31.917147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a015a-912a-4055-bec3-ed88f91341b2 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:31.921309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 538/1079] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:34:31 2022] GET /identity/v3/projects/c172446498ba49dc832907a9acccefa7 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:31.928519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a015a-912a-4055-bec3-ed88f91341b2 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:31.948423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 542/1080] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:34:31 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/1b29c5745a8e42ff8232e5ca22b4a4ad/roles/8a81cd8421c8438cb3a40fd7e56a97eb => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:34:31.961103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:32.046046 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a015a-912a-4055-bec3-ed88f91341b2 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:32.049178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031a79a-b98c-4bf5-83e5-71a6ed75c4df None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:32.086922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693e031-12a1-4c1f-9a0b-c0b0b4715471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:32.087877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693e031-12a1-4c1f-9a0b-c0b0b4715471 None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:32.088322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693e031-12a1-4c1f-9a0b-c0b0b4715471 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:32.088598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693e031-12a1-4c1f-9a0b-c0b0b4715471 None admin] PATH_INFO: `/v3/projects/c172446498ba49dc832907a9acccefa7/users/4daa6ad53cc64ed58c7b8bf7cab56996/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:32.090683 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693e031-12a1-4c1f-9a0b-c0b0b4715471 None admin] RBAC: Authorizing `identity:create_grant(project_id=c172446498ba49dc832907a9acccefa7, user_id=4daa6ad53cc64ed58c7b8bf7cab56996, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:32.092178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693e031-12a1-4c1f-9a0b-c0b0b4715471 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:32.155223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 539/1081] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:34:31 2022] PUT /identity/v3/projects/c172446498ba49dc832907a9acccefa7/users/4daa6ad53cc64ed58c7b8bf7cab56996/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:34:32.160334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031a79a-b98c-4bf5-83e5-71a6ed75c4df None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:32.200690 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f9de0-0c11-4eb3-a675-4ad71cd55291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:32.201360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f9de0-0c11-4eb3-a675-4ad71cd55291 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:32.201665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f9de0-0c11-4eb3-a675-4ad71cd55291 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:32.201936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f9de0-0c11-4eb3-a675-4ad71cd55291 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:32.257404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:32.278431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f9de0-0c11-4eb3-a675-4ad71cd55291 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:32.279897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f9de0-0c11-4eb3-a675-4ad71cd55291 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:32.390716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 543/1082] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:34:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:34:32.396875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693e031-12a1-4c1f-9a0b-c0b0b4715471 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:32.486826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:32.503394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693e031-12a1-4c1f-9a0b-c0b0b4715471 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:32.538502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de7070-2d96-4613-9768-39f8012cae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:32.539282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de7070-2d96-4613-9768-39f8012cae4b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:32.539695 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de7070-2d96-4613-9768-39f8012cae4b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:32.540075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de7070-2d96-4613-9768-39f8012cae4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:32.551195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de7070-2d96-4613-9768-39f8012cae4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:32.553489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de7070-2d96-4613-9768-39f8012cae4b None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-43de7070-2d96-4613-9768-39f8012cae4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:32.568181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:32.569617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:32.573197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:32.573197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:32.573197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:32.573448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 540/1083] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:32.577039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f9de0-0c11-4eb3-a675-4ad71cd55291 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:32.668144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:32.687161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f9de0-0c11-4eb3-a675-4ad71cd55291 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:32.723544 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5acffc-a6f7-448e-a66b-e1a9d488fc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:32.724155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5acffc-a6f7-448e-a66b-e1a9d488fc74 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:32.725234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5acffc-a6f7-448e-a66b-e1a9d488fc74 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:32.725749 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5acffc-a6f7-448e-a66b-e1a9d488fc74 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:32.726427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5acffc-a6f7-448e-a66b-e1a9d488fc74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:32.726797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5acffc-a6f7-448e-a66b-e1a9d488fc74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:32.728224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5acffc-a6f7-448e-a66b-e1a9d488fc74 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:32.745994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 544/1084] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:34.156434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5da7a-02f8-40fe-922d-b315ec8d7166 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:34.157099 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5da7a-02f8-40fe-922d-b315ec8d7166 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:34.157099 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5da7a-02f8-40fe-922d-b315ec8d7166 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:34.158364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 541/1085] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:34.163784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f4a8a-5946-4cfe-9036-6b7b3c5a83c6 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:34.164241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f4a8a-5946-4cfe-9036-6b7b3c5a83c6 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:34.164343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f4a8a-5946-4cfe-9036-6b7b3c5a83c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:34.182867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-764f4a8a-5946-4cfe-9036-6b7b3c5a83c6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:34.184054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-764f4a8a-5946-4cfe-9036-6b7b3c5a83c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:34.254429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 545/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:34.338415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75577eb-9adf-4487-946c-46d9269c9d64 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:34.338697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75577eb-9adf-4487-946c-46d9269c9d64 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:34.338974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75577eb-9adf-4487-946c-46d9269c9d64 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:34.340350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 542/1087] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:34.345937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3793e-d0a1-4d7f-bff1-c80368b65447 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:34.346229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3793e-d0a1-4d7f-bff1-c80368b65447 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:34.346506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3793e-d0a1-4d7f-bff1-c80368b65447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:34.364921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-01e3793e-d0a1-4d7f-bff1-c80368b65447 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:34.366133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-01e3793e-d0a1-4d7f-bff1-c80368b65447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:34.425889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:34.431050 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 546/1088] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:34.531885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75577eb-9adf-4487-946c-46d9269c9d64 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:34.640146 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75577eb-9adf-4487-946c-46d9269c9d64 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:34.672716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d26e1a-4f89-4dc4-8fd9-12945914b161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:34.673383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d26e1a-4f89-4dc4-8fd9-12945914b161 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:34.673659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d26e1a-4f89-4dc4-8fd9-12945914b161 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:34.673925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d26e1a-4f89-4dc4-8fd9-12945914b161 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:34.676534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:34.684789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d26e1a-4f89-4dc4-8fd9-12945914b161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-c8d26e1a-4f89-4dc4-8fd9-12945914b161 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:34:34.689526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:34.690700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:34.691620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 543/1089] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 8 18:34:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:34.696798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3793e-d0a1-4d7f-bff1-c80368b65447 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:34.705114 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d26e1a-4f89-4dc4-8fd9-12945914b161 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:34.797507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3793e-d0a1-4d7f-bff1-c80368b65447 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:34.803404 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d26e1a-4f89-4dc4-8fd9-12945914b161 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:34.834562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c9b58-bb08-4e7d-8047-886a0da6049e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:34.835259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c9b58-bb08-4e7d-8047-886a0da6049e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:34.835544 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c9b58-bb08-4e7d-8047-886a0da6049e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:34.835818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c9b58-bb08-4e7d-8047-886a0da6049e None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:34.836481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c9b58-bb08-4e7d-8047-886a0da6049e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:34.836788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c9b58-bb08-4e7d-8047-886a0da6049e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:34.838029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2137d4-a298-40ae-b9e0-1d57707ebccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:34.838191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c9b58-bb08-4e7d-8047-886a0da6049e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:34.838652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2137d4-a298-40ae-b9e0-1d57707ebccb None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:34.838933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2137d4-a298-40ae-b9e0-1d57707ebccb None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:34.839204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2137d4-a298-40ae-b9e0-1d57707ebccb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:34.851299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 547/1090] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 18:34:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:34.851467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2137d4-a298-40ae-b9e0-1d57707ebccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:34:34.852279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c9b58-bb08-4e7d-8047-886a0da6049e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-4c2137d4-a298-40ae-b9e0-1d57707ebccb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:34:34.854359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:34.855306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:34:34.856229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:34.860302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 544/1091] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:34:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:34:34.861146 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2137d4-a298-40ae-b9e0-1d57707ebccb None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:34.914192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:34.916599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:34.953264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c9b58-bb08-4e7d-8047-886a0da6049e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:34.959170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2137d4-a298-40ae-b9e0-1d57707ebccb None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:34.989025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b091e5-4b6e-4369-b4bd-d119fce5c180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:34.989682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b091e5-4b6e-4369-b4bd-d119fce5c180 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:34.990027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b091e5-4b6e-4369-b4bd-d119fce5c180 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:34.990306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b091e5-4b6e-4369-b4bd-d119fce5c180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:34.994607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed094f4-b945-4f91-ad08-44d3eeaaa32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:34.995214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed094f4-b945-4f91-ad08-44d3eeaaa32e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:34.995498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed094f4-b945-4f91-ad08-44d3eeaaa32e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:34.995773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed094f4-b945-4f91-ad08-44d3eeaaa32e None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:34.996464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed094f4-b945-4f91-ad08-44d3eeaaa32e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:34.996810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed094f4-b945-4f91-ad08-44d3eeaaa32e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:34.998238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed094f4-b945-4f91-ad08-44d3eeaaa32e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:35.001790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b091e5-4b6e-4369-b4bd-d119fce5c180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:35.005897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b091e5-4b6e-4369-b4bd-d119fce5c180 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:35.008025 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 545/1092] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:34:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:35.013837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed094f4-b945-4f91-ad08-44d3eeaaa32e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-39b091e5-4b6e-4369-b4bd-d119fce5c180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:35.018438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:35.019426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:35.020327 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:35.020327 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:35.020327 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:35.020327 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 548/1093] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:35.025805 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b091e5-4b6e-4369-b4bd-d119fce5c180 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:35.122512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed094f4-b945-4f91-ad08-44d3eeaaa32e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:35.126261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b091e5-4b6e-4369-b4bd-d119fce5c180 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:35.139948 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:35.140226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:35.157021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72a00b-bf3f-4a47-8d37-c6b2fbfb1575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:35.157691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72a00b-bf3f-4a47-8d37-c6b2fbfb1575 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:35.157949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72a00b-bf3f-4a47-8d37-c6b2fbfb1575 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:35.158223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72a00b-bf3f-4a47-8d37-c6b2fbfb1575 None admin] PATH_INFO: `/v3/users/4daa6ad53cc64ed58c7b8bf7cab56996` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:35.159353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b72a00b-bf3f-4a47-8d37-c6b2fbfb1575 None admin] RBAC: Authorizing `identity:get_user(user_id=4daa6ad53cc64ed58c7b8bf7cab56996)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:35.159487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e166709-a43f-471a-a0af-28bdc3e4fa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:35.160089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e166709-a43f-471a-a0af-28bdc3e4fa12 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:35.160364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e166709-a43f-471a-a0af-28bdc3e4fa12 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:35.160631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e166709-a43f-471a-a0af-28bdc3e4fa12 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:35.161006 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b72a00b-bf3f-4a47-8d37-c6b2fbfb1575 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:35.161288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e166709-a43f-471a-a0af-28bdc3e4fa12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:35.161589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e166709-a43f-471a-a0af-28bdc3e4fa12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:35.162795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 546/1094] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:34:35 2022] GET /identity/v3/users/4daa6ad53cc64ed58c7b8bf7cab56996 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:35.163044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e166709-a43f-471a-a0af-28bdc3e4fa12 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:35.168717 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72a00b-bf3f-4a47-8d37-c6b2fbfb1575 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:35.175108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 549/1095] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:35.182007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e166709-a43f-471a-a0af-28bdc3e4fa12 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:35.274484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72a00b-bf3f-4a47-8d37-c6b2fbfb1575 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:35.282767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e166709-a43f-471a-a0af-28bdc3e4fa12 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:35.308597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa07d7-db1f-4d87-aef0-b31361ada5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:35.309312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa07d7-db1f-4d87-aef0-b31361ada5af None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:35.309592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa07d7-db1f-4d87-aef0-b31361ada5af None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:35.309870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa07d7-db1f-4d87-aef0-b31361ada5af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:35.317323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2708b3-53b4-4958-bee5-9a1338e644d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:35.318466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2708b3-53b4-4958-bee5-9a1338e644d4 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:35.318842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2708b3-53b4-4958-bee5-9a1338e644d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:35.319133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2708b3-53b4-4958-bee5-9a1338e644d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:35.326029 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2708b3-53b4-4958-bee5-9a1338e644d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:35.327450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2708b3-53b4-4958-bee5-9a1338e644d4 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-3e2708b3-53b4-4958-bee5-9a1338e644d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:34:35.339248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:35.340280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:35.341551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 550/1096] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 8 18:34:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:35.346381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2708b3-53b4-4958-bee5-9a1338e644d4 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:35.364736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:35.377665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa07d7-db1f-4d87-aef0-b31361ada5af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:35.379349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa07d7-db1f-4d87-aef0-b31361ada5af None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:35.381271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:35.445596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2708b3-53b4-4958-bee5-9a1338e644d4 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:35.479158 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 547/1097] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:34:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 18:34:35.480215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6f650-08cf-4a97-9bbe-c14ea41a04c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:35.480783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6f650-08cf-4a97-9bbe-c14ea41a04c1 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:35.481086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6f650-08cf-4a97-9bbe-c14ea41a04c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:35.481365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6f650-08cf-4a97-9bbe-c14ea41a04c1 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:35.482093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6f650-08cf-4a97-9bbe-c14ea41a04c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:35.482404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6f650-08cf-4a97-9bbe-c14ea41a04c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:35.483822 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6f650-08cf-4a97-9bbe-c14ea41a04c1 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:35.484966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa07d7-db1f-4d87-aef0-b31361ada5af None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:35.500976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 551/1098] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 8 18:34:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:35.507185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6f650-08cf-4a97-9bbe-c14ea41a04c1 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:35.572437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:35.587052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa07d7-db1f-4d87-aef0-b31361ada5af None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:35.596855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:35.605998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6f650-08cf-4a97-9bbe-c14ea41a04c1 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:35.624655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4395bd4-16fa-4202-84ad-8911ab5c7b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:35.624655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4395bd4-16fa-4202-84ad-8911ab5c7b94 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:35.624655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4395bd4-16fa-4202-84ad-8911ab5c7b94 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:35.624655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4395bd4-16fa-4202-84ad-8911ab5c7b94 None admin] PATH_INFO: `/v3/projects/c172446498ba49dc832907a9acccefa7` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:35.625898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4395bd4-16fa-4202-84ad-8911ab5c7b94 None admin] RBAC: Authorizing `identity:get_project(project_id=c172446498ba49dc832907a9acccefa7)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:35.627329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4395bd4-16fa-4202-84ad-8911ab5c7b94 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:35.628953 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 548/1099] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:34:35 2022] GET /identity/v3/projects/c172446498ba49dc832907a9acccefa7 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:35.635230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4395bd4-16fa-4202-84ad-8911ab5c7b94 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:35.641101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dcc77-2cf6-471b-99a5-15bfde5ed3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:35.641706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dcc77-2cf6-471b-99a5-15bfde5ed3ec None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:35.641982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dcc77-2cf6-471b-99a5-15bfde5ed3ec None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:35.642251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dcc77-2cf6-471b-99a5-15bfde5ed3ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:35.711412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dcc77-2cf6-471b-99a5-15bfde5ed3ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:35.712864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dcc77-2cf6-471b-99a5-15bfde5ed3ec None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:35.734228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4395bd4-16fa-4202-84ad-8911ab5c7b94 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:35.767837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b150e-db6f-4ff1-94c9-7723cbbb1463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:35.768439 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b150e-db6f-4ff1-94c9-7723cbbb1463 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:35.768723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b150e-db6f-4ff1-94c9-7723cbbb1463 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:35.769000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b150e-db6f-4ff1-94c9-7723cbbb1463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:35.769752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b150e-db6f-4ff1-94c9-7723cbbb1463 None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=c172446498ba49dc832907a9acccefa7, user.id=4daa6ad53cc64ed58c7b8bf7cab56996) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:35.770063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b150e-db6f-4ff1-94c9-7723cbbb1463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:35.772148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b150e-db6f-4ff1-94c9-7723cbbb1463 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:35.777931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 549/1100] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:34:35 2022] GET /identity/v3/role_assignments?user.id=4daa6ad53cc64ed58c7b8bf7cab56996&scope.project.id=c172446498ba49dc832907a9acccefa7&role.id=5c777de40b8449329d714d8c2229115e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:34:35.806651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:35.816742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 552/1101] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:34:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 18:34:35.826049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b150e-db6f-4ff1-94c9-7723cbbb1463 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:35.844991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:35.935753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b150e-db6f-4ff1-94c9-7723cbbb1463 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:35.971042 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18704cb8-e036-4638-8af9-5b28ff613d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:35.971671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18704cb8-e036-4638-8af9-5b28ff613d8d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:35.971946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18704cb8-e036-4638-8af9-5b28ff613d8d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:35.972216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18704cb8-e036-4638-8af9-5b28ff613d8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:35.984522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18704cb8-e036-4638-8af9-5b28ff613d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:35.985989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18704cb8-e036-4638-8af9-5b28ff613d8d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-18704cb8-e036-4638-8af9-5b28ff613d8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:34:36.001386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:36.002447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:34:36.003343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:34:36.003343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:34:36.003343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:36.003343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 550/1102] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:36.008340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dcc77-2cf6-471b-99a5-15bfde5ed3ec None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:36.106940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dcc77-2cf6-471b-99a5-15bfde5ed3ec None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:36.140852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350da41-573b-4091-b6bc-605af21501b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:36.141490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350da41-573b-4091-b6bc-605af21501b2 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:36.141770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350da41-573b-4091-b6bc-605af21501b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:36.142040 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350da41-573b-4091-b6bc-605af21501b2 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:36.142676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350da41-573b-4091-b6bc-605af21501b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:36.142978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350da41-573b-4091-b6bc-605af21501b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:36.144322 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350da41-573b-4091-b6bc-605af21501b2 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:36.160788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 553/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:36.163090 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18704cb8-e036-4638-8af9-5b28ff613d8d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:36.232948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:36.264387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18704cb8-e036-4638-8af9-5b28ff613d8d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:36.298399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dc004-7464-42a4-a7be-672f3a760150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:36.299025 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dc004-7464-42a4-a7be-672f3a760150 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:36.299306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dc004-7464-42a4-a7be-672f3a760150 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:36.299579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dc004-7464-42a4-a7be-672f3a760150 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:36.306828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dc004-7464-42a4-a7be-672f3a760150 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:36.308270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dc004-7464-42a4-a7be-672f3a760150 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-654dc004-7464-42a4-a7be-672f3a760150 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:34:36.319652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:36.320611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:36.321983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:34:36.321983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:34:36.321983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:34:36.321983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:34:36.321983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:34:36.321983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:34:36.321983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:34:36.321983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:34:36.321983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 551/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:34:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:36.326523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350da41-573b-4091-b6bc-605af21501b2 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:36.380753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:36.428276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350da41-573b-4091-b6bc-605af21501b2 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:36.463456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a8476-f6ab-48cc-8a19-f4bd02594f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:36.464058 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a8476-f6ab-48cc-8a19-f4bd02594f72 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:36.464338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a8476-f6ab-48cc-8a19-f4bd02594f72 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:36.464611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a8476-f6ab-48cc-8a19-f4bd02594f72 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:36.465332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a8476-f6ab-48cc-8a19-f4bd02594f72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:36.465640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a8476-f6ab-48cc-8a19-f4bd02594f72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:36.467054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a8476-f6ab-48cc-8a19-f4bd02594f72 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:36.479946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 554/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:34:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:36.486054 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dc004-7464-42a4-a7be-672f3a760150 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:36.585854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dc004-7464-42a4-a7be-672f3a760150 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:36.621547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:36.627287 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3614b-80ba-41b1-be2c-a1350766bb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:36.627892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3614b-80ba-41b1-be2c-a1350766bb90 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:36.628169 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3614b-80ba-41b1-be2c-a1350766bb90 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:36.628438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3614b-80ba-41b1-be2c-a1350766bb90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:36.629187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3614b-80ba-41b1-be2c-a1350766bb90 None admin] RBAC: Adding query filter params (role.id=8a81cd8421c8438cb3a40fd7e56a97eb, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=1b29c5745a8e42ff8232e5ca22b4a4ad) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:36.629492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3614b-80ba-41b1-be2c-a1350766bb90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:36.631768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3614b-80ba-41b1-be2c-a1350766bb90 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:36.637489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 552/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:34:36 2022] GET /identity/v3/role_assignments?user.id=1b29c5745a8e42ff8232e5ca22b4a4ad&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=8a81cd8421c8438cb3a40fd7e56a97eb => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:37.991381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07a58d-a583-44d5-9f77-28b0b95afefd None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:37.991906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07a58d-a583-44d5-9f77-28b0b95afefd None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:37.991906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07a58d-a583-44d5-9f77-28b0b95afefd None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:37.993232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 555/1107] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:37.998542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d5ef3-70f4-413b-bd30-320be20cff06 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:37.998950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d5ef3-70f4-413b-bd30-320be20cff06 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:37.999099 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d5ef3-70f4-413b-bd30-320be20cff06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:38.017862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-1e7d5ef3-70f4-413b-bd30-320be20cff06 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:38.019049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1e7d5ef3-70f4-413b-bd30-320be20cff06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:38.084852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 553/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:38.364457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07a58d-a583-44d5-9f77-28b0b95afefd None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:38.463966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07a58d-a583-44d5-9f77-28b0b95afefd None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:38.477549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:38.498602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e06853-6433-4b4c-9aba-8faf5f226202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:38.499223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e06853-6433-4b4c-9aba-8faf5f226202 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:38.499505 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e06853-6433-4b4c-9aba-8faf5f226202 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:38.499781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e06853-6433-4b4c-9aba-8faf5f226202 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:38.500890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e06853-6433-4b4c-9aba-8faf5f226202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:38.502343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e06853-6433-4b4c-9aba-8faf5f226202 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:38.503901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 556/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:38.509786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d5ef3-70f4-413b-bd30-320be20cff06 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:38.609230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d5ef3-70f4-413b-bd30-320be20cff06 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:38.637903 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:38.648477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b20b5-a5c5-4933-b2ab-3095dc562e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:38.649130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b20b5-a5c5-4933-b2ab-3095dc562e76 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:38.649387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b20b5-a5c5-4933-b2ab-3095dc562e76 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:38.649661 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b20b5-a5c5-4933-b2ab-3095dc562e76 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:38.650315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b20b5-a5c5-4933-b2ab-3095dc562e76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:38.651734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b20b5-a5c5-4933-b2ab-3095dc562e76 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:38.753757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 554/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:34:38 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:34:40.933887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69b293-1d53-477b-a5f6-567d3489dc42 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:40.934409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69b293-1d53-477b-a5f6-567d3489dc42 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:40.934409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69b293-1d53-477b-a5f6-567d3489dc42 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:40.935827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 557/1111] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:40.940962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53dc26-fbb4-47fe-b336-9ce7001acabe None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:40.941435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53dc26-fbb4-47fe-b336-9ce7001acabe None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:40.941545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53dc26-fbb4-47fe-b336-9ce7001acabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:40.959892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ba53dc26-fbb4-47fe-b336-9ce7001acabe None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:40.961085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ba53dc26-fbb4-47fe-b336-9ce7001acabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:41.026260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 555/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:41.299295 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69b293-1d53-477b-a5f6-567d3489dc42 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:41.399606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69b293-1d53-477b-a5f6-567d3489dc42 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:41.433546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5faab06-6ada-4f9e-a97e-2eeecf46b61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:41.434166 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5faab06-6ada-4f9e-a97e-2eeecf46b61e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:41.434447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5faab06-6ada-4f9e-a97e-2eeecf46b61e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:41.434721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5faab06-6ada-4f9e-a97e-2eeecf46b61e None admin] PATH_INFO: `/v3/roles/6b3a5d54d6044e9b8d8270e7ff3145e8` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:41.436107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5faab06-6ada-4f9e-a97e-2eeecf46b61e None admin] RBAC: Authorizing `identity:get_role(role_id=6b3a5d54d6044e9b8d8270e7ff3145e8)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:41.437498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5faab06-6ada-4f9e-a97e-2eeecf46b61e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:41.438802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 558/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:34:41 2022] GET /identity/v3/roles/6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:41.444981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53dc26-fbb4-47fe-b336-9ce7001acabe None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:41.541005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53dc26-fbb4-47fe-b336-9ce7001acabe None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:41.551962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:41.574471 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd70f03-258a-4756-b011-fa36226780b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:41.575081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd70f03-258a-4756-b011-fa36226780b5 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:41.575359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd70f03-258a-4756-b011-fa36226780b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:41.575632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd70f03-258a-4756-b011-fa36226780b5 None admin] PATH_INFO: `/v3/users/aa389cb8e33a4aed8be50904289d1964` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:41.588536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd70f03-258a-4756-b011-fa36226780b5 None admin] RBAC: Authorizing `identity:get_user(user_id=aa389cb8e33a4aed8be50904289d1964)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:41.590416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd70f03-258a-4756-b011-fa36226780b5 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:41.592041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 556/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:34:41 2022] GET /identity/v3/users/aa389cb8e33a4aed8be50904289d1964 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:41.598176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5faab06-6ada-4f9e-a97e-2eeecf46b61e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:41.647098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:41.696699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5faab06-6ada-4f9e-a97e-2eeecf46b61e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:41.729237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f680b-1d8b-4a5f-9508-a2cc3c7324b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:41.729829 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f680b-1d8b-4a5f-9508-a2cc3c7324b8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:41.730106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f680b-1d8b-4a5f-9508-a2cc3c7324b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:41.730376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f680b-1d8b-4a5f-9508-a2cc3c7324b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:41.795726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f680b-1d8b-4a5f-9508-a2cc3c7324b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:41.797176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f680b-1d8b-4a5f-9508-a2cc3c7324b8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:41.842701 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:41.892834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 559/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:34:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:34:41.898709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd70f03-258a-4756-b011-fa36226780b5 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:41.995117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd70f03-258a-4756-b011-fa36226780b5 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:42.028096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980890c-a6c3-4701-aef3-9da294248281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:42.028721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980890c-a6c3-4701-aef3-9da294248281 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:42.029001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980890c-a6c3-4701-aef3-9da294248281 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:42.029306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980890c-a6c3-4701-aef3-9da294248281 None admin] PATH_INFO: `/v3/projects/c172446498ba49dc832907a9acccefa7` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:42.030422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980890c-a6c3-4701-aef3-9da294248281 None admin] RBAC: Authorizing `identity:get_project(project_id=c172446498ba49dc832907a9acccefa7)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:42.031836 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980890c-a6c3-4701-aef3-9da294248281 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:42.033479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 557/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:34:41 2022] GET /identity/v3/projects/c172446498ba49dc832907a9acccefa7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:42.040800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f680b-1d8b-4a5f-9508-a2cc3c7324b8 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:42.140354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f680b-1d8b-4a5f-9508-a2cc3c7324b8 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:42.174617 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3d1de-a250-4349-8b03-fef61d2a0b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:42.175207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d1de-a250-4349-8b03-fef61d2a0b35 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:42.175487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d1de-a250-4349-8b03-fef61d2a0b35 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:42.175760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d1de-a250-4349-8b03-fef61d2a0b35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:42.176471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3d1de-a250-4349-8b03-fef61d2a0b35 None admin] RBAC: Adding query filter params (role.id=6b3a5d54d6044e9b8d8270e7ff3145e8, scope.project.id=c172446498ba49dc832907a9acccefa7, user.id=aa389cb8e33a4aed8be50904289d1964) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:42.176779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3d1de-a250-4349-8b03-fef61d2a0b35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:42.179461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3d1de-a250-4349-8b03-fef61d2a0b35 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:42.184788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 560/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:34:42 2022] GET /identity/v3/role_assignments?user.id=aa389cb8e33a4aed8be50904289d1964&scope.project.id=c172446498ba49dc832907a9acccefa7&role.id=6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:44.341658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a21cb-66cb-4b59-bf49-8cc4ebcc42c3 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:44.342183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a21cb-66cb-4b59-bf49-8cc4ebcc42c3 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:44.342276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a21cb-66cb-4b59-bf49-8cc4ebcc42c3 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:44.343553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 558/1118] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:44.349168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee18c8-9e73-45ac-86fe-00c8696eab02 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:44.349543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee18c8-9e73-45ac-86fe-00c8696eab02 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:44.349720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee18c8-9e73-45ac-86fe-00c8696eab02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:44.368027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7fee18c8-9e73-45ac-86fe-00c8696eab02 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:44.369263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7fee18c8-9e73-45ac-86fe-00c8696eab02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:44.376358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:44.435210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 561/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:44.711748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a21cb-66cb-4b59-bf49-8cc4ebcc42c3 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:44.748217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:44.813187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a21cb-66cb-4b59-bf49-8cc4ebcc42c3 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:44.846481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825445c-0e4b-49f5-b562-bbefaa730908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:44.847101 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825445c-0e4b-49f5-b562-bbefaa730908 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:44.847384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825445c-0e4b-49f5-b562-bbefaa730908 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:44.847659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825445c-0e4b-49f5-b562-bbefaa730908 None admin] PATH_INFO: `/v3/roles/6b3a5d54d6044e9b8d8270e7ff3145e8` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:44.849135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825445c-0e4b-49f5-b562-bbefaa730908 None admin] RBAC: Authorizing `identity:get_role(role_id=6b3a5d54d6044e9b8d8270e7ff3145e8)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:44.850492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825445c-0e4b-49f5-b562-bbefaa730908 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:44.851801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 559/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:34:44 2022] GET /identity/v3/roles/6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:44.857664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee18c8-9e73-45ac-86fe-00c8696eab02 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:44.954072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee18c8-9e73-45ac-86fe-00c8696eab02 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:44.986875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6c420-f7c3-49e2-95dd-50791f2a5fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:44.987488 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6c420-f7c3-49e2-95dd-50791f2a5fc3 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:44.987764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6c420-f7c3-49e2-95dd-50791f2a5fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:44.988033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6c420-f7c3-49e2-95dd-50791f2a5fc3 None admin] PATH_INFO: `/v3/users/aa389cb8e33a4aed8be50904289d1964` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:44.989159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6c420-f7c3-49e2-95dd-50791f2a5fc3 None admin] RBAC: Authorizing `identity:get_user(user_id=aa389cb8e33a4aed8be50904289d1964)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:44.990756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6c420-f7c3-49e2-95dd-50791f2a5fc3 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:44.992303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 562/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:34:44 2022] GET /identity/v3/users/aa389cb8e33a4aed8be50904289d1964 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:44.997996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825445c-0e4b-49f5-b562-bbefaa730908 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:45.095167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825445c-0e4b-49f5-b562-bbefaa730908 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:45.097434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:45.127426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a25f-a4a6-4501-8769-d6aa2e1a3077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:45.127998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a25f-a4a6-4501-8769-d6aa2e1a3077 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:45.128257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a25f-a4a6-4501-8769-d6aa2e1a3077 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:45.128527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a25f-a4a6-4501-8769-d6aa2e1a3077 None admin] PATH_INFO: `/v3/projects/c172446498ba49dc832907a9acccefa7` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:45.129686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59a25f-a4a6-4501-8769-d6aa2e1a3077 None admin] RBAC: Authorizing `identity:get_project(project_id=c172446498ba49dc832907a9acccefa7)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:45.131624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59a25f-a4a6-4501-8769-d6aa2e1a3077 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:45.133493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 560/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:34:44 2022] GET /identity/v3/projects/c172446498ba49dc832907a9acccefa7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:45.139049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6c420-f7c3-49e2-95dd-50791f2a5fc3 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:45.151856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:45.242160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6c420-f7c3-49e2-95dd-50791f2a5fc3 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:45.275918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491704cf-90d5-4224-a045-d341faa3540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:45.276569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491704cf-90d5-4224-a045-d341faa3540c None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:45.276848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491704cf-90d5-4224-a045-d341faa3540c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:45.277153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491704cf-90d5-4224-a045-d341faa3540c None admin] PATH_INFO: `/v3/projects/c172446498ba49dc832907a9acccefa7/users/aa389cb8e33a4aed8be50904289d1964/roles/6b3a5d54d6044e9b8d8270e7ff3145e8` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:45.279258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491704cf-90d5-4224-a045-d341faa3540c None admin] RBAC: Authorizing `identity:create_grant(project_id=c172446498ba49dc832907a9acccefa7, user_id=aa389cb8e33a4aed8be50904289d1964, role_id=6b3a5d54d6044e9b8d8270e7ff3145e8)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:45.280668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491704cf-90d5-4224-a045-d341faa3540c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:45.339422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 563/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:34:45 2022] PUT /identity/v3/projects/c172446498ba49dc832907a9acccefa7/users/aa389cb8e33a4aed8be50904289d1964/roles/6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:34:47.464046 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957baa4-9d90-45a2-96d8-8b30d2b71b0e None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:47.464577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957baa4-9d90-45a2-96d8-8b30d2b71b0e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:47.464577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957baa4-9d90-45a2-96d8-8b30d2b71b0e None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:47.465973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 561/1124] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:47.471319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6ca6e-013d-4e74-a37c-9abf9ffa21be None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:47.471757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6ca6e-013d-4e74-a37c-9abf9ffa21be None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:47.471859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6ca6e-013d-4e74-a37c-9abf9ffa21be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:47.490751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-acb6ca6e-013d-4e74-a37c-9abf9ffa21be None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:47.491924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-acb6ca6e-013d-4e74-a37c-9abf9ffa21be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:47.553253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:47.557739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 564/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:47.826307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957baa4-9d90-45a2-96d8-8b30d2b71b0e None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:47.928590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957baa4-9d90-45a2-96d8-8b30d2b71b0e None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:47.961503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8fba6-0c3e-4b51-bed7-ca9171ea8749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:47.962115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8fba6-0c3e-4b51-bed7-ca9171ea8749 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:47.962394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8fba6-0c3e-4b51-bed7-ca9171ea8749 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:47.962664 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8fba6-0c3e-4b51-bed7-ca9171ea8749 None admin] PATH_INFO: `/v3/roles/6b3a5d54d6044e9b8d8270e7ff3145e8` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:47.964052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8fba6-0c3e-4b51-bed7-ca9171ea8749 None admin] RBAC: Authorizing `identity:get_role(role_id=6b3a5d54d6044e9b8d8270e7ff3145e8)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:47.965401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8fba6-0c3e-4b51-bed7-ca9171ea8749 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:47.966708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 562/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:34:47 2022] GET /identity/v3/roles/6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:47.972982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6ca6e-013d-4e74-a37c-9abf9ffa21be None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:48.070169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6ca6e-013d-4e74-a37c-9abf9ffa21be None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:48.104720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752260fd-761e-42a1-a4d6-ff5e46f4979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:48.105310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752260fd-761e-42a1-a4d6-ff5e46f4979a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:48.108880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752260fd-761e-42a1-a4d6-ff5e46f4979a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:48.108880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752260fd-761e-42a1-a4d6-ff5e46f4979a None admin] PATH_INFO: `/v3/users/aa389cb8e33a4aed8be50904289d1964` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:48.108880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752260fd-761e-42a1-a4d6-ff5e46f4979a None admin] RBAC: Authorizing `identity:get_user(user_id=aa389cb8e33a4aed8be50904289d1964)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:48.108880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752260fd-761e-42a1-a4d6-ff5e46f4979a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:48.109927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 565/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:34:47 2022] GET /identity/v3/users/aa389cb8e33a4aed8be50904289d1964 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:48.116346 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8fba6-0c3e-4b51-bed7-ca9171ea8749 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:48.157064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:48.213701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8fba6-0c3e-4b51-bed7-ca9171ea8749 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:48.246839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6deb6-2455-444d-80e5-d2c60a07ddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:48.247426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6deb6-2455-444d-80e5-d2c60a07ddd6 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:48.247703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6deb6-2455-444d-80e5-d2c60a07ddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:48.247972 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6deb6-2455-444d-80e5-d2c60a07ddd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:48.312933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6deb6-2455-444d-80e5-d2c60a07ddd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:48.314392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6deb6-2455-444d-80e5-d2c60a07ddd6 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:48.353889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:48.409464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 563/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:34:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:34:48.415507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752260fd-761e-42a1-a4d6-ff5e46f4979a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:48.481473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:48.514175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752260fd-761e-42a1-a4d6-ff5e46f4979a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:48.546686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a32dea-d1b2-49fb-9f12-9c46c0d1e97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:48.547303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a32dea-d1b2-49fb-9f12-9c46c0d1e97c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:48.547581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a32dea-d1b2-49fb-9f12-9c46c0d1e97c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:48.547849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a32dea-d1b2-49fb-9f12-9c46c0d1e97c None admin] PATH_INFO: `/v3/projects/c172446498ba49dc832907a9acccefa7` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:48.548939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a32dea-d1b2-49fb-9f12-9c46c0d1e97c None admin] RBAC: Authorizing `identity:get_project(project_id=c172446498ba49dc832907a9acccefa7)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:48.550372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a32dea-d1b2-49fb-9f12-9c46c0d1e97c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:48.552494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 566/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:34:48 2022] GET /identity/v3/projects/c172446498ba49dc832907a9acccefa7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:48.559189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6deb6-2455-444d-80e5-d2c60a07ddd6 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:48.661619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6deb6-2455-444d-80e5-d2c60a07ddd6 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:48.695710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2b6bc-e859-415c-86ed-70fa5b38419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:48.696306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2b6bc-e859-415c-86ed-70fa5b38419d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:48.696585 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2b6bc-e859-415c-86ed-70fa5b38419d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:48.696856 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2b6bc-e859-415c-86ed-70fa5b38419d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:48.697604 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2b6bc-e859-415c-86ed-70fa5b38419d None admin] RBAC: Adding query filter params (role.id=6b3a5d54d6044e9b8d8270e7ff3145e8, scope.project.id=c172446498ba49dc832907a9acccefa7, user.id=aa389cb8e33a4aed8be50904289d1964) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:48.697912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2b6bc-e859-415c-86ed-70fa5b38419d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:48.699327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2b6bc-e859-415c-86ed-70fa5b38419d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:48.704928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 564/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:34:48 2022] GET /identity/v3/role_assignments?user.id=aa389cb8e33a4aed8be50904289d1964&scope.project.id=c172446498ba49dc832907a9acccefa7&role.id=6b3a5d54d6044e9b8d8270e7ff3145e8 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:50.888896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880d4ff-fb73-4a02-8517-3587be0c7ce8 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:50.889460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880d4ff-fb73-4a02-8517-3587be0c7ce8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:50.889460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880d4ff-fb73-4a02-8517-3587be0c7ce8 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:50.890769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 567/1131] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:50.896659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5b77f-2de2-4b3c-b5f3-0cb27890a24e None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:50.897161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5b77f-2de2-4b3c-b5f3-0cb27890a24e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:50.897449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5b77f-2de2-4b3c-b5f3-0cb27890a24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:50.904687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:50.916347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-4db5b77f-2de2-4b3c-b5f3-0cb27890a24e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:50.917589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4db5b77f-2de2-4b3c-b5f3-0cb27890a24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:50.983488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 565/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:51.249589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880d4ff-fb73-4a02-8517-3587be0c7ce8 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:51.352569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880d4ff-fb73-4a02-8517-3587be0c7ce8 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:51.386020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b430b-5aaf-4860-ad97-f25e4ac693d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:51.386642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b430b-5aaf-4860-ad97-f25e4ac693d3 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:51.386921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b430b-5aaf-4860-ad97-f25e4ac693d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:51.387193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b430b-5aaf-4860-ad97-f25e4ac693d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:51.388286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b430b-5aaf-4860-ad97-f25e4ac693d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:51.389725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b430b-5aaf-4860-ad97-f25e4ac693d3 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:51.391287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 568/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:51.397340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5b77f-2de2-4b3c-b5f3-0cb27890a24e None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:51.494667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5b77f-2de2-4b3c-b5f3-0cb27890a24e None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:51.529007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977efdf4-5b33-49b8-bab0-7f5d942918e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:51.529670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977efdf4-5b33-49b8-bab0-7f5d942918e7 None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:51.530139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977efdf4-5b33-49b8-bab0-7f5d942918e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:51.530604 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977efdf4-5b33-49b8-bab0-7f5d942918e7 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:51.531265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977efdf4-5b33-49b8-bab0-7f5d942918e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:51.532684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977efdf4-5b33-49b8-bab0-7f5d942918e7 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:51.537380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:51.623539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 566/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:34:51 2022] POST /identity/v3/projects => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:34:53.820714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec881864-02ab-4241-9f12-c0631b11b366 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:53.821362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec881864-02ab-4241-9f12-c0631b11b366 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:53.821362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec881864-02ab-4241-9f12-c0631b11b366 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:53.822638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 569/1135] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:53.827892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95bd66-fe4d-41ab-b417-25b3f94a64a6 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:53.828300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95bd66-fe4d-41ab-b417-25b3f94a64a6 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:53.828440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95bd66-fe4d-41ab-b417-25b3f94a64a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:53.846430 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ac95bd66-fe4d-41ab-b417-25b3f94a64a6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:53.847598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ac95bd66-fe4d-41ab-b417-25b3f94a64a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:53.912836 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 567/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:54.179271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec881864-02ab-4241-9f12-c0631b11b366 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:54.183059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:54.276847 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec881864-02ab-4241-9f12-c0631b11b366 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:54.309882 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fbaec-3106-4d56-a532-27e8f4cfbe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:54.310499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fbaec-3106-4d56-a532-27e8f4cfbe40 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:54.310776 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fbaec-3106-4d56-a532-27e8f4cfbe40 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:54.311043 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fbaec-3106-4d56-a532-27e8f4cfbe40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:54.312124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fbaec-3106-4d56-a532-27e8f4cfbe40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:54.313572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fbaec-3106-4d56-a532-27e8f4cfbe40 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:54.315117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 570/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:34:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:54.321395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95bd66-fe4d-41ab-b417-25b3f94a64a6 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:54.417102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95bd66-fe4d-41ab-b417-25b3f94a64a6 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:54.432959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:54.449851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf30cc-61a0-4da0-a963-61854bf2fbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:54.450450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf30cc-61a0-4da0-a963-61854bf2fbdb None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:54.450726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf30cc-61a0-4da0-a963-61854bf2fbdb None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:54.450997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf30cc-61a0-4da0-a963-61854bf2fbdb None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:54.451676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf30cc-61a0-4da0-a963-61854bf2fbdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:54.453118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bf30cc-61a0-4da0-a963-61854bf2fbdb None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:54.528081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 568/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:34:54 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:34:56.709531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119fe63-2cbe-41a3-8eb6-f912d480a235 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:56.710055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119fe63-2cbe-41a3-8eb6-f912d480a235 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:56.710055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119fe63-2cbe-41a3-8eb6-f912d480a235 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:56.711429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 571/1139] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:34:56.716860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114eae40-2a03-41b3-937e-fbd4a5c24881 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:56.717304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114eae40-2a03-41b3-937e-fbd4a5c24881 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:56.717448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114eae40-2a03-41b3-937e-fbd4a5c24881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:56.736164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-114eae40-2a03-41b3-937e-fbd4a5c24881 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:34:56.737388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-114eae40-2a03-41b3-937e-fbd4a5c24881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:34:56.801873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 569/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:34:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:34:57.079168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119fe63-2cbe-41a3-8eb6-f912d480a235 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:57.150201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:57.178456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119fe63-2cbe-41a3-8eb6-f912d480a235 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:57.211577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41be90-eb5e-4ebf-8310-b6f540653d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:57.212186 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41be90-eb5e-4ebf-8310-b6f540653d5e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:57.212465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41be90-eb5e-4ebf-8310-b6f540653d5e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:57.212735 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41be90-eb5e-4ebf-8310-b6f540653d5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:57.222789 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41be90-eb5e-4ebf-8310-b6f540653d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-9f41be90-eb5e-4ebf-8310-b6f540653d5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:34:57.228349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:34:57.229517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:34:57.230573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 572/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:34:57.235621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114eae40-2a03-41b3-937e-fbd4a5c24881 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:57.334866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114eae40-2a03-41b3-937e-fbd4a5c24881 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:57.336256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:57.368822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432015-76ff-4152-b366-c3fb0f6f5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:57.369450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432015-76ff-4152-b366-c3fb0f6f5306 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:57.369734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432015-76ff-4152-b366-c3fb0f6f5306 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:57.370007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432015-76ff-4152-b366-c3fb0f6f5306 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:57.370672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a432015-76ff-4152-b366-c3fb0f6f5306 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:57.370982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a432015-76ff-4152-b366-c3fb0f6f5306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:57.372345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a432015-76ff-4152-b366-c3fb0f6f5306 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:57.380322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 570/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:57.386338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41be90-eb5e-4ebf-8310-b6f540653d5e None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:57.485155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41be90-eb5e-4ebf-8310-b6f540653d5e None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:57.514177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:57.518933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fc829-4338-422b-8cb8-d406b26ee3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:57.523939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fc829-4338-422b-8cb8-d406b26ee3aa None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:57.523939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fc829-4338-422b-8cb8-d406b26ee3aa None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:57.523939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fc829-4338-422b-8cb8-d406b26ee3aa None admin] PATH_INFO: `/v3/users/00d26a82e760494e8867e8773970e594` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:57.533563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fc829-4338-422b-8cb8-d406b26ee3aa None admin] RBAC: Authorizing `identity:get_user(user_id=00d26a82e760494e8867e8773970e594)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:57.534989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fc829-4338-422b-8cb8-d406b26ee3aa None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:57.537320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 573/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:34:57 2022] GET /identity/v3/users/00d26a82e760494e8867e8773970e594 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:57.543193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432015-76ff-4152-b366-c3fb0f6f5306 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:57.642595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432015-76ff-4152-b366-c3fb0f6f5306 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:57.676040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cfd63-cb42-43d9-824c-7098ef93465e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:57.676636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cfd63-cb42-43d9-824c-7098ef93465e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:57.676993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cfd63-cb42-43d9-824c-7098ef93465e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:57.677362 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cfd63-cb42-43d9-824c-7098ef93465e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:57.706892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:57.743427 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cfd63-cb42-43d9-824c-7098ef93465e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:57.744852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11cfd63-cb42-43d9-824c-7098ef93465e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:57.838739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 571/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:34:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:34:57.844963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fc829-4338-422b-8cb8-d406b26ee3aa None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:57.942755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fc829-4338-422b-8cb8-d406b26ee3aa None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:57.975782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5730b-a512-4566-a109-116462d0e489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:57.976411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5730b-a512-4566-a109-116462d0e489 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:57.976691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5730b-a512-4566-a109-116462d0e489 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:57.976963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5730b-a512-4566-a109-116462d0e489 None admin] PATH_INFO: `/v3/projects/bff3b3f4f80241ea87eb3b1c5d746e0f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:57.978104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5730b-a512-4566-a109-116462d0e489 None admin] RBAC: Authorizing `identity:get_project(project_id=bff3b3f4f80241ea87eb3b1c5d746e0f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:57.979559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5730b-a512-4566-a109-116462d0e489 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:57.981209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 574/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:34:57 2022] GET /identity/v3/projects/bff3b3f4f80241ea87eb3b1c5d746e0f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:34:57.987217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cfd63-cb42-43d9-824c-7098ef93465e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:34:58.063214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:34:58.103354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cfd63-cb42-43d9-824c-7098ef93465e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:34:58.137528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882531fd-db6b-4f4a-8a4b-9f9f53f5eca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:34:58.138121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882531fd-db6b-4f4a-8a4b-9f9f53f5eca3 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:34:58.138402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882531fd-db6b-4f4a-8a4b-9f9f53f5eca3 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:34:58.138676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882531fd-db6b-4f4a-8a4b-9f9f53f5eca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:34:58.139418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882531fd-db6b-4f4a-8a4b-9f9f53f5eca3 None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=bff3b3f4f80241ea87eb3b1c5d746e0f, user.id=00d26a82e760494e8867e8773970e594) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:34:58.139728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882531fd-db6b-4f4a-8a4b-9f9f53f5eca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:34:58.141168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882531fd-db6b-4f4a-8a4b-9f9f53f5eca3 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:34:58.146420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 572/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:34:57 2022] GET /identity/v3/role_assignments?user.id=00d26a82e760494e8867e8773970e594&scope.project.id=bff3b3f4f80241ea87eb3b1c5d746e0f&role.id=5c777de40b8449329d714d8c2229115e => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:00.259796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a6553-2dbc-41ba-8c31-3033e7059ae5 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:00.260331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a6553-2dbc-41ba-8c31-3033e7059ae5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:00.260331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a6553-2dbc-41ba-8c31-3033e7059ae5 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:00.261696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 575/1147] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:35:00.267175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f9f6-79d7-4658-b9c8-2e8385aab0a9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:00.267594 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f9f6-79d7-4658-b9c8-2e8385aab0a9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:00.267703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1f9f6-79d7-4658-b9c8-2e8385aab0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:00.286451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-6ad1f9f6-79d7-4658-b9c8-2e8385aab0a9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:35:00.287831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6ad1f9f6-79d7-4658-b9c8-2e8385aab0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:35:00.353804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 573/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:35:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:35:00.626172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a6553-2dbc-41ba-8c31-3033e7059ae5 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:00.685527 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:00.732056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a6553-2dbc-41ba-8c31-3033e7059ae5 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:00.766111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e0818-15bc-40f7-a2fe-5b3c82f0b684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:00.766736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e0818-15bc-40f7-a2fe-5b3c82f0b684 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:00.767018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e0818-15bc-40f7-a2fe-5b3c82f0b684 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:00.767293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e0818-15bc-40f7-a2fe-5b3c82f0b684 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:00.777649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e0818-15bc-40f7-a2fe-5b3c82f0b684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-191e0818-15bc-40f7-a2fe-5b3c82f0b684 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:35:00.782429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:35:00.783494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:35:00.784530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 576/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:35:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:35:00.790833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1f9f6-79d7-4658-b9c8-2e8385aab0a9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:00.848074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:00.889515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1f9f6-79d7-4658-b9c8-2e8385aab0a9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:00.922459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54804ab2-b549-493c-93d4-543920b37e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:00.923077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804ab2-b549-493c-93d4-543920b37e1d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:00.923339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804ab2-b549-493c-93d4-543920b37e1d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:00.923612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804ab2-b549-493c-93d4-543920b37e1d None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:00.924270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54804ab2-b549-493c-93d4-543920b37e1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:35:00.924573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54804ab2-b549-493c-93d4-543920b37e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:00.925958 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54804ab2-b549-493c-93d4-543920b37e1d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:00.934896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 574/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:35:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:00.940819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e0818-15bc-40f7-a2fe-5b3c82f0b684 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:01.039407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e0818-15bc-40f7-a2fe-5b3c82f0b684 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:01.057088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:01.073216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05406296-6149-4161-9a2b-1c2aa88f4102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:01.073841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05406296-6149-4161-9a2b-1c2aa88f4102 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:01.074120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05406296-6149-4161-9a2b-1c2aa88f4102 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:01.074391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05406296-6149-4161-9a2b-1c2aa88f4102 None admin] PATH_INFO: `/v3/users/00d26a82e760494e8867e8773970e594` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:01.075490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05406296-6149-4161-9a2b-1c2aa88f4102 None admin] RBAC: Authorizing `identity:get_user(user_id=00d26a82e760494e8867e8773970e594)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:01.076904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05406296-6149-4161-9a2b-1c2aa88f4102 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:01.078528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 577/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:35:00 2022] GET /identity/v3/users/00d26a82e760494e8867e8773970e594 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:01.084986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54804ab2-b549-493c-93d4-543920b37e1d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:01.181400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54804ab2-b549-493c-93d4-543920b37e1d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:01.200310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:01.214178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8242d60-6c10-4e0e-a7b0-21314fcc0895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:01.214792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8242d60-6c10-4e0e-a7b0-21314fcc0895 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:01.215071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8242d60-6c10-4e0e-a7b0-21314fcc0895 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:01.215341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8242d60-6c10-4e0e-a7b0-21314fcc0895 None admin] PATH_INFO: `/v3/projects/bff3b3f4f80241ea87eb3b1c5d746e0f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:01.216436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8242d60-6c10-4e0e-a7b0-21314fcc0895 None admin] RBAC: Authorizing `identity:get_project(project_id=bff3b3f4f80241ea87eb3b1c5d746e0f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:01.217919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8242d60-6c10-4e0e-a7b0-21314fcc0895 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:01.219790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 575/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:35:01 2022] GET /identity/v3/projects/bff3b3f4f80241ea87eb3b1c5d746e0f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:01.226021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05406296-6149-4161-9a2b-1c2aa88f4102 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:01.326528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05406296-6149-4161-9a2b-1c2aa88f4102 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:01.360132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60468f-8229-45fa-b177-06aeb999a17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:01.360777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60468f-8229-45fa-b177-06aeb999a17d None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:01.361071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60468f-8229-45fa-b177-06aeb999a17d None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:01.361353 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60468f-8229-45fa-b177-06aeb999a17d None admin] PATH_INFO: `/v3/projects/bff3b3f4f80241ea87eb3b1c5d746e0f/users/00d26a82e760494e8867e8773970e594/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:01.363214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60468f-8229-45fa-b177-06aeb999a17d None admin] RBAC: Authorizing `identity:create_grant(project_id=bff3b3f4f80241ea87eb3b1c5d746e0f, user_id=00d26a82e760494e8867e8773970e594, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:01.365176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60468f-8229-45fa-b177-06aeb999a17d None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:01.450822 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 578/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:35:01 2022] PUT /identity/v3/projects/bff3b3f4f80241ea87eb3b1c5d746e0f/users/00d26a82e760494e8867e8773970e594/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:35:03.569877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb7fb3-b858-4caa-b26c-ad6cf3956214 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:03.570441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb7fb3-b858-4caa-b26c-ad6cf3956214 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:03.570441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb7fb3-b858-4caa-b26c-ad6cf3956214 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:03.571787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 576/1154] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:35:03.576874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6e65c-1519-4a55-8e73-59fab1e23b65 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:03.577293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6e65c-1519-4a55-8e73-59fab1e23b65 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:03.577447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6e65c-1519-4a55-8e73-59fab1e23b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:03.596447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-96a6e65c-1519-4a55-8e73-59fab1e23b65 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:35:03.597658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-96a6e65c-1519-4a55-8e73-59fab1e23b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:35:03.639431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:03.666972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 579/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:35:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:35:03.944003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb7fb3-b858-4caa-b26c-ad6cf3956214 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:04.052671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb7fb3-b858-4caa-b26c-ad6cf3956214 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:04.086400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a2718-02a4-47de-921d-e6ae4d2f0306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:04.087009 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a2718-02a4-47de-921d-e6ae4d2f0306 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:04.087288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a2718-02a4-47de-921d-e6ae4d2f0306 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:04.087558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a2718-02a4-47de-921d-e6ae4d2f0306 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:04.097672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a2718-02a4-47de-921d-e6ae4d2f0306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-280a2718-02a4-47de-921d-e6ae4d2f0306 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:35:04.102488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:35:04.103603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:35:04.104662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 577/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:35:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:35:04.109824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6e65c-1519-4a55-8e73-59fab1e23b65 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:04.207635 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6e65c-1519-4a55-8e73-59fab1e23b65 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:04.240741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6386a-5bab-4a98-9de1-1ce160e41173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:04.241363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6386a-5bab-4a98-9de1-1ce160e41173 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:04.241640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6386a-5bab-4a98-9de1-1ce160e41173 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:04.241909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6386a-5bab-4a98-9de1-1ce160e41173 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:04.242563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6386a-5bab-4a98-9de1-1ce160e41173 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:35:04.242864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6386a-5bab-4a98-9de1-1ce160e41173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:04.244209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6386a-5bab-4a98-9de1-1ce160e41173 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:04.252384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 580/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:35:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:04.259265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a2718-02a4-47de-921d-e6ae4d2f0306 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:04.316638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:04.367366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a2718-02a4-47de-921d-e6ae4d2f0306 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:04.399420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2251b9-468b-40a1-b310-c202b5c55a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:04.400032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2251b9-468b-40a1-b310-c202b5c55a6d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:04.400307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2251b9-468b-40a1-b310-c202b5c55a6d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:04.400573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2251b9-468b-40a1-b310-c202b5c55a6d None admin] PATH_INFO: `/v3/users/00d26a82e760494e8867e8773970e594` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:04.401697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2251b9-468b-40a1-b310-c202b5c55a6d None admin] RBAC: Authorizing `identity:get_user(user_id=00d26a82e760494e8867e8773970e594)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:04.403109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2251b9-468b-40a1-b310-c202b5c55a6d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:04.404666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 578/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:35:04 2022] GET /identity/v3/users/00d26a82e760494e8867e8773970e594 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:04.410483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6386a-5bab-4a98-9de1-1ce160e41173 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:04.450158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:04.509397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6386a-5bab-4a98-9de1-1ce160e41173 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:04.542984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a42de-5aa8-467d-b688-8576f6e80bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:04.543585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a42de-5aa8-467d-b688-8576f6e80bb1 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:04.543863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a42de-5aa8-467d-b688-8576f6e80bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:04.544134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a42de-5aa8-467d-b688-8576f6e80bb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:04.617194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a42de-5aa8-467d-b688-8576f6e80bb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:04.618694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a42de-5aa8-467d-b688-8576f6e80bb1 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:04.661711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:04.721055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 581/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:35:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:35:04.727035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2251b9-468b-40a1-b310-c202b5c55a6d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:04.829369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2251b9-468b-40a1-b310-c202b5c55a6d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:04.845731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:04.863086 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef621ccb-3443-441b-bfb6-fda056b6d833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:04.863706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef621ccb-3443-441b-bfb6-fda056b6d833 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:04.863990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef621ccb-3443-441b-bfb6-fda056b6d833 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:04.864793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef621ccb-3443-441b-bfb6-fda056b6d833 None admin] PATH_INFO: `/v3/projects/bff3b3f4f80241ea87eb3b1c5d746e0f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:04.865942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef621ccb-3443-441b-bfb6-fda056b6d833 None admin] RBAC: Authorizing `identity:get_project(project_id=bff3b3f4f80241ea87eb3b1c5d746e0f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:04.867420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef621ccb-3443-441b-bfb6-fda056b6d833 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:04.869034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 579/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:35:04 2022] GET /identity/v3/projects/bff3b3f4f80241ea87eb3b1c5d746e0f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:04.875387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a42de-5aa8-467d-b688-8576f6e80bb1 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:04.972818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a42de-5aa8-467d-b688-8576f6e80bb1 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:05.005481 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52289b2e-e153-4144-90ce-1490900e5e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:05.006115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289b2e-e153-4144-90ce-1490900e5e4e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:05.006393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289b2e-e153-4144-90ce-1490900e5e4e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:05.006661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52289b2e-e153-4144-90ce-1490900e5e4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:05.007365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52289b2e-e153-4144-90ce-1490900e5e4e None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=bff3b3f4f80241ea87eb3b1c5d746e0f, user.id=00d26a82e760494e8867e8773970e594) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:35:05.007668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52289b2e-e153-4144-90ce-1490900e5e4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:05.009092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52289b2e-e153-4144-90ce-1490900e5e4e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:05.014608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 582/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:35:04 2022] GET /identity/v3/role_assignments?user.id=00d26a82e760494e8867e8773970e594&scope.project.id=bff3b3f4f80241ea87eb3b1c5d746e0f&role.id=5c777de40b8449329d714d8c2229115e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:07.203951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77da76-3ae9-496e-9951-b7e72c4130ef None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:07.204502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77da76-3ae9-496e-9951-b7e72c4130ef None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:07.204502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77da76-3ae9-496e-9951-b7e72c4130ef None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:07.205889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 580/1162] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:35:07.211681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a45143-52af-4fa6-9db0-776f8f486460 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:07.212064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a45143-52af-4fa6-9db0-776f8f486460 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:07.212225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a45143-52af-4fa6-9db0-776f8f486460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:07.218525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:07.231376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f8a45143-52af-4fa6-9db0-776f8f486460 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:35:07.238266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f8a45143-52af-4fa6-9db0-776f8f486460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:35:07.298450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 583/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:35:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:35:07.575697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77da76-3ae9-496e-9951-b7e72c4130ef None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:07.678333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77da76-3ae9-496e-9951-b7e72c4130ef None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:07.711507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec40e7-8478-4d19-b052-6719847d9fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:07.712100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec40e7-8478-4d19-b052-6719847d9fb8 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:07.712381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec40e7-8478-4d19-b052-6719847d9fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:07.712651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec40e7-8478-4d19-b052-6719847d9fb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:07.763241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:07.778786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec40e7-8478-4d19-b052-6719847d9fb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:07.780224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec40e7-8478-4d19-b052-6719847d9fb8 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:07.874343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 581/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:35:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:35:07.880318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a45143-52af-4fa6-9db0-776f8f486460 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:07.975557 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a45143-52af-4fa6-9db0-776f8f486460 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:08.003711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:08.009005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6f3d8-4ede-46e9-84ed-d23b8a3917e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:08.014859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6f3d8-4ede-46e9-84ed-d23b8a3917e0 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:08.014859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6f3d8-4ede-46e9-84ed-d23b8a3917e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:08.014859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6f3d8-4ede-46e9-84ed-d23b8a3917e0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:08.016728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6f3d8-4ede-46e9-84ed-d23b8a3917e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:08.018180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6f3d8-4ede-46e9-84ed-d23b8a3917e0 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-0ee6f3d8-4ede-46e9-84ed-d23b8a3917e0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:35:08.029157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:35:08.030402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:35:08.031510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 584/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 18:35:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:35:08.036187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec40e7-8478-4d19-b052-6719847d9fb8 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:08.117961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:08.133809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec40e7-8478-4d19-b052-6719847d9fb8 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:08.166809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ca0db-c3a2-44c4-998f-a184e0db01ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:08.167402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca0db-c3a2-44c4-998f-a184e0db01ef None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:08.167681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca0db-c3a2-44c4-998f-a184e0db01ef None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:08.167957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca0db-c3a2-44c4-998f-a184e0db01ef None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:08.168590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ca0db-c3a2-44c4-998f-a184e0db01ef None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:35:08.168894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ca0db-c3a2-44c4-998f-a184e0db01ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:08.170364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ca0db-c3a2-44c4-998f-a184e0db01ef None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:08.177902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 582/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 8 18:35:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:10.292425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31673f91-2ae5-415c-b8ca-9d26ba506000 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:10.292990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31673f91-2ae5-415c-b8ca-9d26ba506000 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:10.293123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31673f91-2ae5-415c-b8ca-9d26ba506000 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:10.294443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 585/1167] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:35:10.299614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d4c98-7cd4-4922-9bd7-367d81c3cbb5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:10.300044 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d4c98-7cd4-4922-9bd7-367d81c3cbb5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:10.300155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d4c98-7cd4-4922-9bd7-367d81c3cbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:10.320153 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-7e4d4c98-7cd4-4922-9bd7-367d81c3cbb5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:35:10.321392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7e4d4c98-7cd4-4922-9bd7-367d81c3cbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:35:10.387531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 583/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:35:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 18:35:10.656098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31673f91-2ae5-415c-b8ca-9d26ba506000 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:10.756476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31673f91-2ae5-415c-b8ca-9d26ba506000 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:10.789136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e8153-ec7c-4848-bd87-41c5a714b4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:10.789722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e8153-ec7c-4848-bd87-41c5a714b4ae None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:10.789998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e8153-ec7c-4848-bd87-41c5a714b4ae None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:10.790268 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e8153-ec7c-4848-bd87-41c5a714b4ae None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:10.790931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e8153-ec7c-4848-bd87-41c5a714b4ae None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:10.792275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e8153-ec7c-4848-bd87-41c5a714b4ae None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:10.871889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 586/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Thu Sep 8 18:35:10 2022] POST /identity/v3/domains => generated 267 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:35:13.043542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694be54-e088-4235-8358-83f6cda07f5f None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:13.044069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694be54-e088-4235-8358-83f6cda07f5f None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:13.044163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694be54-e088-4235-8358-83f6cda07f5f None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:13.045487 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 584/1170] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:35:13.050596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cf0ea-6ccb-4963-a6e6-a31abf02e23b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:13.050988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cf0ea-6ccb-4963-a6e6-a31abf02e23b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:13.051141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cf0ea-6ccb-4963-a6e6-a31abf02e23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:13.069424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8c2cf0ea-6ccb-4963-a6e6-a31abf02e23b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:35:13.070593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8c2cf0ea-6ccb-4963-a6e6-a31abf02e23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:35:13.118514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:13.137853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 587/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:35:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:35:13.415049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694be54-e088-4235-8358-83f6cda07f5f None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:13.499925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:13.522724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694be54-e088-4235-8358-83f6cda07f5f None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:13.556779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37178f1-efea-4f88-ade2-3fefb9399776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:13.557433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37178f1-efea-4f88-ade2-3fefb9399776 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:13.557713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37178f1-efea-4f88-ade2-3fefb9399776 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:13.557985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37178f1-efea-4f88-ade2-3fefb9399776 None admin] PATH_INFO: `/v3/domains/6177ef3e3bed4a0299d253a01d43dd5e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:13.568704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37178f1-efea-4f88-ade2-3fefb9399776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6177ef3e3bed4a0299d253a01d43dd5e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:13.570165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37178f1-efea-4f88-ade2-3fefb9399776 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:13.571742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 585/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Sep 8 18:35:13 2022] GET /identity/v3/domains/6177ef3e3bed4a0299d253a01d43dd5e => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:13.577666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cf0ea-6ccb-4963-a6e6-a31abf02e23b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:13.675490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cf0ea-6ccb-4963-a6e6-a31abf02e23b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:13.709248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbb425-5560-44ff-bbae-d3ff08a8905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:13.709842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbb425-5560-44ff-bbae-d3ff08a8905a None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:13.710122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbb425-5560-44ff-bbae-d3ff08a8905a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:13.710397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbb425-5560-44ff-bbae-d3ff08a8905a None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:13.711051 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fbb425-5560-44ff-bbae-d3ff08a8905a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:13.712468 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fbb425-5560-44ff-bbae-d3ff08a8905a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:13.797707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 588/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:35:13 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:35:15.954427 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274c3d5-399f-44f8-bb81-0fa0275e4aab None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:15.954951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274c3d5-399f-44f8-bb81-0fa0275e4aab None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:15.954951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274c3d5-399f-44f8-bb81-0fa0275e4aab None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:15.956237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 586/1174] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:35:15.961893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d773-53b1-4f2d-a887-225058d4aa3b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:15.962308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d773-53b1-4f2d-a887-225058d4aa3b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:15.962448 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d773-53b1-4f2d-a887-225058d4aa3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:15.981346 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2ed4d773-53b1-4f2d-a887-225058d4aa3b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:35:15.982560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2ed4d773-53b1-4f2d-a887-225058d4aa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:35:16.008923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:16.047842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 589/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:35:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:35:16.312379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274c3d5-399f-44f8-bb81-0fa0275e4aab None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:16.412579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274c3d5-399f-44f8-bb81-0fa0275e4aab None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:16.445929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1d10d-3e3d-42f3-8c89-4466b45995ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:16.446543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1d10d-3e3d-42f3-8c89-4466b45995ef None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:16.446822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1d10d-3e3d-42f3-8c89-4466b45995ef None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:16.447096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1d10d-3e3d-42f3-8c89-4466b45995ef None admin] PATH_INFO: `/v3/domains/6177ef3e3bed4a0299d253a01d43dd5e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:16.448196 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1d10d-3e3d-42f3-8c89-4466b45995ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=6177ef3e3bed4a0299d253a01d43dd5e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:16.449650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1d10d-3e3d-42f3-8c89-4466b45995ef None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:16.451197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 587/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Sep 8 18:35:16 2022] GET /identity/v3/domains/6177ef3e3bed4a0299d253a01d43dd5e => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:16.456941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4d773-53b1-4f2d-a887-225058d4aa3b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:16.554239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4d773-53b1-4f2d-a887-225058d4aa3b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:16.586291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b2448-4a1d-42c6-bbec-659ac15b29bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:16.586821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b2448-4a1d-42c6-bbec-659ac15b29bc None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:16.587097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b2448-4a1d-42c6-bbec-659ac15b29bc None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:16.587366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b2448-4a1d-42c6-bbec-659ac15b29bc None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:16.588010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b2448-4a1d-42c6-bbec-659ac15b29bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:16.589460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b2448-4a1d-42c6-bbec-659ac15b29bc None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:16.707925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 590/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:35:16 2022] POST /identity/v3/users => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:35:18.891214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326cbe5-4fd2-4fb3-8809-1b9598fe4a92 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:18.891718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326cbe5-4fd2-4fb3-8809-1b9598fe4a92 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:18.891718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326cbe5-4fd2-4fb3-8809-1b9598fe4a92 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:18.893081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 588/1178] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:35:18.898858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5a00d-bd7e-4900-8f56-30f06b2d7d81 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:18.899246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5a00d-bd7e-4900-8f56-30f06b2d7d81 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:18.899410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5a00d-bd7e-4900-8f56-30f06b2d7d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:18.917754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a2e5a00d-bd7e-4900-8f56-30f06b2d7d81 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:35:18.918941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a2e5a00d-bd7e-4900-8f56-30f06b2d7d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:35:18.933026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:18.994139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 591/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:35:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:35:19.303415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326cbe5-4fd2-4fb3-8809-1b9598fe4a92 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:19.313686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:19.429887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326cbe5-4fd2-4fb3-8809-1b9598fe4a92 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:19.470194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da3c33-b118-47a6-89c1-f6b2a031e931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:19.470964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da3c33-b118-47a6-89c1-f6b2a031e931 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:19.471244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da3c33-b118-47a6-89c1-f6b2a031e931 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:19.471518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da3c33-b118-47a6-89c1-f6b2a031e931 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:19.484407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da3c33-b118-47a6-89c1-f6b2a031e931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-e9da3c33-b118-47a6-89c1-f6b2a031e931 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:35:19.490409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:35:19.491467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:35:19.492593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 589/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:35:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:35:19.499961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5a00d-bd7e-4900-8f56-30f06b2d7d81 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:19.625116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5a00d-bd7e-4900-8f56-30f06b2d7d81 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:19.668625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d94e15-a005-4f0c-9ad3-db5033b9883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:19.669268 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d94e15-a005-4f0c-9ad3-db5033b9883f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:19.669547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d94e15-a005-4f0c-9ad3-db5033b9883f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:19.669820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d94e15-a005-4f0c-9ad3-db5033b9883f None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:19.670480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d94e15-a005-4f0c-9ad3-db5033b9883f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:35:19.670779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d94e15-a005-4f0c-9ad3-db5033b9883f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:19.672145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d94e15-a005-4f0c-9ad3-db5033b9883f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:19.681472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 592/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:35:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:19.688714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da3c33-b118-47a6-89c1-f6b2a031e931 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:19.764968 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:19.813232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da3c33-b118-47a6-89c1-f6b2a031e931 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:19.855835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694b7ad-c153-426f-b994-4d7dcda3fc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:19.856515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694b7ad-c153-426f-b994-4d7dcda3fc09 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:19.856746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694b7ad-c153-426f-b994-4d7dcda3fc09 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:19.857019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694b7ad-c153-426f-b994-4d7dcda3fc09 None admin] PATH_INFO: `/v3/users/df261a854eb34da5b101c24744d3c718` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:19.875217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694b7ad-c153-426f-b994-4d7dcda3fc09 None admin] RBAC: Authorizing `identity:get_user(user_id=df261a854eb34da5b101c24744d3c718)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:19.876775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694b7ad-c153-426f-b994-4d7dcda3fc09 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:19.878505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 590/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:35:19 2022] GET /identity/v3/users/df261a854eb34da5b101c24744d3c718 => generated 311 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:19.885293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d94e15-a005-4f0c-9ad3-db5033b9883f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:19.902582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:20.007568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d94e15-a005-4f0c-9ad3-db5033b9883f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:20.039843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abd5b3-fe7d-4b64-bb70-4bdebf9bca0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:20.040439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abd5b3-fe7d-4b64-bb70-4bdebf9bca0a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:20.040716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abd5b3-fe7d-4b64-bb70-4bdebf9bca0a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:20.040986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abd5b3-fe7d-4b64-bb70-4bdebf9bca0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:20.105724 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abd5b3-fe7d-4b64-bb70-4bdebf9bca0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:20.107188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abd5b3-fe7d-4b64-bb70-4bdebf9bca0a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:20.120360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:20.203183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 593/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:35:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:35:20.209938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694b7ad-c153-426f-b994-4d7dcda3fc09 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:20.315388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694b7ad-c153-426f-b994-4d7dcda3fc09 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:20.338717 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:20.349283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744f39a-b0df-4562-819a-ab3fe0742ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:20.349904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f39a-b0df-4562-819a-ab3fe0742ee7 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:20.350183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f39a-b0df-4562-819a-ab3fe0742ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:20.359801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f39a-b0df-4562-819a-ab3fe0742ee7 None admin] PATH_INFO: `/v3/projects/fd80b397493941d09b8bb7aca88b8220` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:20.359801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744f39a-b0df-4562-819a-ab3fe0742ee7 None admin] RBAC: Authorizing `identity:get_project(project_id=fd80b397493941d09b8bb7aca88b8220)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:20.359801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744f39a-b0df-4562-819a-ab3fe0742ee7 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:20.359801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 591/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:35:20 2022] GET /identity/v3/projects/fd80b397493941d09b8bb7aca88b8220 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:20.360670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abd5b3-fe7d-4b64-bb70-4bdebf9bca0a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:20.458432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abd5b3-fe7d-4b64-bb70-4bdebf9bca0a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:20.470860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:20.491205 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d370c9c-e311-4179-b6de-649c117f4bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:20.491802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d370c9c-e311-4179-b6de-649c117f4bf9 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:20.492082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d370c9c-e311-4179-b6de-649c117f4bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:20.492352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d370c9c-e311-4179-b6de-649c117f4bf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:20.493087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d370c9c-e311-4179-b6de-649c117f4bf9 None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=fd80b397493941d09b8bb7aca88b8220, user.id=df261a854eb34da5b101c24744d3c718) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:35:20.493404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d370c9c-e311-4179-b6de-649c117f4bf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:20.494805 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d370c9c-e311-4179-b6de-649c117f4bf9 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:20.500302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 594/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:35:20 2022] GET /identity/v3/role_assignments?user.id=df261a854eb34da5b101c24744d3c718&scope.project.id=fd80b397493941d09b8bb7aca88b8220&role.id=5c777de40b8449329d714d8c2229115e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:22.599236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57735b-ae43-4d92-83f8-981805c4af16 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:22.599760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57735b-ae43-4d92-83f8-981805c4af16 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:22.599899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57735b-ae43-4d92-83f8-981805c4af16 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:22.601572 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 592/1186] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:35:22.606923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d31e24-9a43-46e2-bcb7-e66734861c38 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:22.607299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d31e24-9a43-46e2-bcb7-e66734861c38 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:22.607462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d31e24-9a43-46e2-bcb7-e66734861c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:22.625828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-16d31e24-9a43-46e2-bcb7-e66734861c38 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:35:22.627001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-16d31e24-9a43-46e2-bcb7-e66734861c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:35:22.691359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 595/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:35:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:35:22.959822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57735b-ae43-4d92-83f8-981805c4af16 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:23.058066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57735b-ae43-4d92-83f8-981805c4af16 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:23.091096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1dd09-2049-4257-b0d7-a6b7bfd72ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:23.091722 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1dd09-2049-4257-b0d7-a6b7bfd72ac7 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:23.092005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1dd09-2049-4257-b0d7-a6b7bfd72ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:23.092283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1dd09-2049-4257-b0d7-a6b7bfd72ac7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:23.103201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1dd09-2049-4257-b0d7-a6b7bfd72ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-2fa1dd09-2049-4257-b0d7-a6b7bfd72ac7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:35:23.108001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:35:23.109129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:35:23.109987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 593/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:35:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:35:23.115429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d31e24-9a43-46e2-bcb7-e66734861c38 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:23.213381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d31e24-9a43-46e2-bcb7-e66734861c38 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:23.220693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:23.249600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab12e0-7b58-4483-aa2b-a148baf80c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:23.250209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab12e0-7b58-4483-aa2b-a148baf80c39 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:23.250491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab12e0-7b58-4483-aa2b-a148baf80c39 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:23.250766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab12e0-7b58-4483-aa2b-a148baf80c39 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:23.251428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab12e0-7b58-4483-aa2b-a148baf80c39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:35:23.251736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab12e0-7b58-4483-aa2b-a148baf80c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:23.253116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab12e0-7b58-4483-aa2b-a148baf80c39 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:23.262427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 596/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:35:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:23.268264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1dd09-2049-4257-b0d7-a6b7bfd72ac7 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:23.321513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:23.368172 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1dd09-2049-4257-b0d7-a6b7bfd72ac7 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:23.402191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e625877-11a3-43c2-a89e-4ba458baeb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:23.402804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625877-11a3-43c2-a89e-4ba458baeb44 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:23.403084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625877-11a3-43c2-a89e-4ba458baeb44 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:23.403356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625877-11a3-43c2-a89e-4ba458baeb44 None admin] PATH_INFO: `/v3/users/df261a854eb34da5b101c24744d3c718` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:23.404460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e625877-11a3-43c2-a89e-4ba458baeb44 None admin] RBAC: Authorizing `identity:get_user(user_id=df261a854eb34da5b101c24744d3c718)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:23.405908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e625877-11a3-43c2-a89e-4ba458baeb44 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:23.407461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 594/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:35:23 2022] GET /identity/v3/users/df261a854eb34da5b101c24744d3c718 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:23.413757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab12e0-7b58-4483-aa2b-a148baf80c39 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:23.511628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab12e0-7b58-4483-aa2b-a148baf80c39 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:23.544693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d951a-3250-475b-8789-ee989b155e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:23.545345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d951a-3250-475b-8789-ee989b155e4a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:23.545623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d951a-3250-475b-8789-ee989b155e4a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:23.545893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d951a-3250-475b-8789-ee989b155e4a None admin] PATH_INFO: `/v3/projects/fd80b397493941d09b8bb7aca88b8220` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:23.546992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d951a-3250-475b-8789-ee989b155e4a None admin] RBAC: Authorizing `identity:get_project(project_id=fd80b397493941d09b8bb7aca88b8220)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:23.548392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d951a-3250-475b-8789-ee989b155e4a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:23.550034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 597/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:35:23 2022] GET /identity/v3/projects/fd80b397493941d09b8bb7aca88b8220 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:23.556097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e625877-11a3-43c2-a89e-4ba458baeb44 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:23.656341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e625877-11a3-43c2-a89e-4ba458baeb44 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:23.675086 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:23.688849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3375689-e53e-4f36-b3a1-88c17067e0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:23.689703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3375689-e53e-4f36-b3a1-88c17067e0b6 None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:23.689988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3375689-e53e-4f36-b3a1-88c17067e0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:23.690265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3375689-e53e-4f36-b3a1-88c17067e0b6 None admin] PATH_INFO: `/v3/projects/fd80b397493941d09b8bb7aca88b8220/users/df261a854eb34da5b101c24744d3c718/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:23.692422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3375689-e53e-4f36-b3a1-88c17067e0b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=fd80b397493941d09b8bb7aca88b8220, user_id=df261a854eb34da5b101c24744d3c718, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:23.693879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3375689-e53e-4f36-b3a1-88c17067e0b6 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:23.753139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 595/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:35:23 2022] PUT /identity/v3/projects/fd80b397493941d09b8bb7aca88b8220/users/df261a854eb34da5b101c24744d3c718/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:35:25.936929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761d10-ff9b-48b4-916c-065b453c9c30 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:25.937478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761d10-ff9b-48b4-916c-065b453c9c30 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:25.937478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761d10-ff9b-48b4-916c-065b453c9c30 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:25.938794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 598/1193] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:35:25.944690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9dc53-008e-4112-9d9f-74f3660c1b21 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:25.947224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9dc53-008e-4112-9d9f-74f3660c1b21 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:25.947224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9dc53-008e-4112-9d9f-74f3660c1b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:25.963155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-08c9dc53-008e-4112-9d9f-74f3660c1b21 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:35:25.964326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-08c9dc53-008e-4112-9d9f-74f3660c1b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:35:26.029453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 596/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:35:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:35:26.307950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06761d10-ff9b-48b4-916c-065b453c9c30 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:26.345651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:26.409085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06761d10-ff9b-48b4-916c-065b453c9c30 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:26.441583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e7cf7-e212-4bdd-980b-0d0e0ff9ec0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:26.442983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e7cf7-e212-4bdd-980b-0d0e0ff9ec0f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:26.443262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e7cf7-e212-4bdd-980b-0d0e0ff9ec0f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:26.443535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e7cf7-e212-4bdd-980b-0d0e0ff9ec0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:26.453462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e7cf7-e212-4bdd-980b-0d0e0ff9ec0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-191e7cf7-e212-4bdd-980b-0d0e0ff9ec0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:35:26.458733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:35:26.459858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:35:26.460765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 599/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:35:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:35:26.465880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9dc53-008e-4112-9d9f-74f3660c1b21 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:26.563081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9dc53-008e-4112-9d9f-74f3660c1b21 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:26.574176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:26.597644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe225de5-79e5-40f2-a36c-80d939bf640d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:26.598442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe225de5-79e5-40f2-a36c-80d939bf640d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:26.598950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe225de5-79e5-40f2-a36c-80d939bf640d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:26.599282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe225de5-79e5-40f2-a36c-80d939bf640d None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:26.599951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe225de5-79e5-40f2-a36c-80d939bf640d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:35:26.602431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe225de5-79e5-40f2-a36c-80d939bf640d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:26.602431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe225de5-79e5-40f2-a36c-80d939bf640d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:26.611031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 597/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:35:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:26.617785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e7cf7-e212-4bdd-980b-0d0e0ff9ec0f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:26.710362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:26.715882 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e7cf7-e212-4bdd-980b-0d0e0ff9ec0f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:26.749879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033b678-89e4-424c-8f05-06260cfa5cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:26.750493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033b678-89e4-424c-8f05-06260cfa5cd9 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:26.750773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033b678-89e4-424c-8f05-06260cfa5cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:26.751043 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033b678-89e4-424c-8f05-06260cfa5cd9 None admin] PATH_INFO: `/v3/users/df261a854eb34da5b101c24744d3c718` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:26.752153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033b678-89e4-424c-8f05-06260cfa5cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=df261a854eb34da5b101c24744d3c718)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:26.753595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033b678-89e4-424c-8f05-06260cfa5cd9 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:26.755698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 600/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 8 18:35:26 2022] GET /identity/v3/users/df261a854eb34da5b101c24744d3c718 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:26.761528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe225de5-79e5-40f2-a36c-80d939bf640d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:26.860181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe225de5-79e5-40f2-a36c-80d939bf640d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:26.893165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33450c14-ac7e-4db1-8ee8-d3c626b7c7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:26.893769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33450c14-ac7e-4db1-8ee8-d3c626b7c7f6 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:26.894052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33450c14-ac7e-4db1-8ee8-d3c626b7c7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:26.894334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33450c14-ac7e-4db1-8ee8-d3c626b7c7f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:26.936519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:26.963455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33450c14-ac7e-4db1-8ee8-d3c626b7c7f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:26.964908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33450c14-ac7e-4db1-8ee8-d3c626b7c7f6 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:27.060652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 598/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:35:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:35:27.066648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033b678-89e4-424c-8f05-06260cfa5cd9 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:27.166343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033b678-89e4-424c-8f05-06260cfa5cd9 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:27.200542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d5f61-a3de-4695-b674-bc9096dd85ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:27.201330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d5f61-a3de-4695-b674-bc9096dd85ee None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:27.201618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d5f61-a3de-4695-b674-bc9096dd85ee None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:27.201895 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d5f61-a3de-4695-b674-bc9096dd85ee None admin] PATH_INFO: `/v3/projects/fd80b397493941d09b8bb7aca88b8220` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:27.203002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d5f61-a3de-4695-b674-bc9096dd85ee None admin] RBAC: Authorizing `identity:get_project(project_id=fd80b397493941d09b8bb7aca88b8220)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:27.204419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d5f61-a3de-4695-b674-bc9096dd85ee None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:27.206059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 601/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 8 18:35:27 2022] GET /identity/v3/projects/fd80b397493941d09b8bb7aca88b8220 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:27.212856 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33450c14-ac7e-4db1-8ee8-d3c626b7c7f6 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:35:27.281771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:27.310069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33450c14-ac7e-4db1-8ee8-d3c626b7c7f6 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:35:27.343557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78e26c-6cba-4cc0-9710-04b1d05d336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:35:27.344148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e26c-6cba-4cc0-9710-04b1d05d336d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:35:27.344433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e26c-6cba-4cc0-9710-04b1d05d336d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:35:27.344961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e26c-6cba-4cc0-9710-04b1d05d336d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:35:27.345704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78e26c-6cba-4cc0-9710-04b1d05d336d None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=fd80b397493941d09b8bb7aca88b8220, user.id=df261a854eb34da5b101c24744d3c718) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:35:27.346015 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78e26c-6cba-4cc0-9710-04b1d05d336d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:35:27.347438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78e26c-6cba-4cc0-9710-04b1d05d336d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:35:27.353543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 599/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:35:27 2022] GET /identity/v3/role_assignments?user.id=df261a854eb34da5b101c24744d3c718&scope.project.id=fd80b397493941d09b8bb7aca88b8220&role.id=5c777de40b8449329d714d8c2229115e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:35:37.196465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=84 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:35:37.350888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=32 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:40.359578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e9325-cda6-429f-8257-5ed9f284fb0d None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:40.360189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e9325-cda6-429f-8257-5ed9f284fb0d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:40.360189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e9325-cda6-429f-8257-5ed9f284fb0d None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:40.361515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 602/1201] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:36:40.366711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b340383-5c21-4ee7-a5ef-acc393885af7 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:40.367161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b340383-5c21-4ee7-a5ef-acc393885af7 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:40.367263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b340383-5c21-4ee7-a5ef-acc393885af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:40.386050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-7b340383-5c21-4ee7-a5ef-acc393885af7 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:36:40.387289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7b340383-5c21-4ee7-a5ef-acc393885af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:36:40.456132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 600/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:36:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:36:40.723868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e9325-cda6-429f-8257-5ed9f284fb0d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:40.823816 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e9325-cda6-429f-8257-5ed9f284fb0d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:40.857883 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69332e0d-9a09-4c12-86e1-36c6f2255dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:40.858502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69332e0d-9a09-4c12-86e1-36c6f2255dce None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:40.858778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69332e0d-9a09-4c12-86e1-36c6f2255dce None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:40.859045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69332e0d-9a09-4c12-86e1-36c6f2255dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:40.869172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69332e0d-9a09-4c12-86e1-36c6f2255dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:40.872826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69332e0d-9a09-4c12-86e1-36c6f2255dce None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-69332e0d-9a09-4c12-86e1-36c6f2255dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:36:40.885243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:40.886451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:36:40.887723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:36:40.887723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:40.887723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:36:40.887723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 603/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:40.892479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b340383-5c21-4ee7-a5ef-acc393885af7 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:40.992661 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b340383-5c21-4ee7-a5ef-acc393885af7 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:41.020841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:41.027414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b3bc5-ea66-47ce-9ec8-06cdc6a1553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:41.028015 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3bc5-ea66-47ce-9ec8-06cdc6a1553c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:41.028291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3bc5-ea66-47ce-9ec8-06cdc6a1553c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:41.028560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b3bc5-ea66-47ce-9ec8-06cdc6a1553c None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:41.029231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b3bc5-ea66-47ce-9ec8-06cdc6a1553c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:41.029534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b3bc5-ea66-47ce-9ec8-06cdc6a1553c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:41.030888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b3bc5-ea66-47ce-9ec8-06cdc6a1553c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:41.042417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 601/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:41.048196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69332e0d-9a09-4c12-86e1-36c6f2255dce None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:41.100948 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:41.158154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69332e0d-9a09-4c12-86e1-36c6f2255dce None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:41.190436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d7f43-43e4-4a79-8b33-cbfa78d93a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:41.191123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7f43-43e4-4a79-8b33-cbfa78d93a1c None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:41.191402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7f43-43e4-4a79-8b33-cbfa78d93a1c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:41.191671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d7f43-43e4-4a79-8b33-cbfa78d93a1c None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:41.192311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d7f43-43e4-4a79-8b33-cbfa78d93a1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:41.193747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d7f43-43e4-4a79-8b33-cbfa78d93a1c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:41.289758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 604/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 8 18:36:41 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:36:43.483309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada12cc7-17ae-41ca-bcfa-7547bd9f9366 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:43.483873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada12cc7-17ae-41ca-bcfa-7547bd9f9366 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:43.483873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada12cc7-17ae-41ca-bcfa-7547bd9f9366 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:43.485259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 602/1206] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:36:43.490710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b978712-d392-4ca4-b1fc-4023773545ea None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:43.491119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b978712-d392-4ca4-b1fc-4023773545ea None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:43.491260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b978712-d392-4ca4-b1fc-4023773545ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:43.510653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-6b978712-d392-4ca4-b1fc-4023773545ea None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:36:43.511853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6b978712-d392-4ca4-b1fc-4023773545ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:36:43.579419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 605/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:36:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:36:43.853617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada12cc7-17ae-41ca-bcfa-7547bd9f9366 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:43.955637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada12cc7-17ae-41ca-bcfa-7547bd9f9366 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:43.989937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cc22d-f7e0-4ef0-ba4c-0eee9cd13a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:43.990553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cc22d-f7e0-4ef0-ba4c-0eee9cd13a6b None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:43.990828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cc22d-f7e0-4ef0-ba4c-0eee9cd13a6b None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:43.991094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cc22d-f7e0-4ef0-ba4c-0eee9cd13a6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:44.001248 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cc22d-f7e0-4ef0-ba4c-0eee9cd13a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-8b9cc22d-f7e0-4ef0-ba4c-0eee9cd13a6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:36:44.006050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:44.007085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:44.008360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 603/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:36:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:44.013374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b978712-d392-4ca4-b1fc-4023773545ea None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:44.057316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:44.114233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b978712-d392-4ca4-b1fc-4023773545ea None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:44.148542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656fc99-f5d1-498a-98ca-174bf3375972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:44.149194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656fc99-f5d1-498a-98ca-174bf3375972 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:44.150073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656fc99-f5d1-498a-98ca-174bf3375972 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:44.150341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656fc99-f5d1-498a-98ca-174bf3375972 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:44.150999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656fc99-f5d1-498a-98ca-174bf3375972 None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:44.151310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656fc99-f5d1-498a-98ca-174bf3375972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:44.152712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656fc99-f5d1-498a-98ca-174bf3375972 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:44.160354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 606/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:36:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:44.166274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cc22d-f7e0-4ef0-ba4c-0eee9cd13a6b None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:44.217464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:44.269877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cc22d-f7e0-4ef0-ba4c-0eee9cd13a6b None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:44.303638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307570c-7356-450d-ad1d-b4ccd64fa0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:44.304285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307570c-7356-450d-ad1d-b4ccd64fa0da None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:44.304566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307570c-7356-450d-ad1d-b4ccd64fa0da None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:44.304838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307570c-7356-450d-ad1d-b4ccd64fa0da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:44.315521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307570c-7356-450d-ad1d-b4ccd64fa0da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:44.318666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307570c-7356-450d-ad1d-b4ccd64fa0da None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-c307570c-7356-450d-ad1d-b4ccd64fa0da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:36:44.332452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:44.338255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:36:44.339522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:36:44.339522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:44.339522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:44.339522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 604/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:44.340282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656fc99-f5d1-498a-98ca-174bf3375972 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:44.438876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:44.441790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656fc99-f5d1-498a-98ca-174bf3375972 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:44.476246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec74a9-745b-4541-b19c-2cb6a290849f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:44.476848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec74a9-745b-4541-b19c-2cb6a290849f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:44.477157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec74a9-745b-4541-b19c-2cb6a290849f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:44.478430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec74a9-745b-4541-b19c-2cb6a290849f None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:44.479074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec74a9-745b-4541-b19c-2cb6a290849f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:44.479383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec74a9-745b-4541-b19c-2cb6a290849f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:44.480731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec74a9-745b-4541-b19c-2cb6a290849f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:44.491687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 607/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:44.497997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307570c-7356-450d-ad1d-b4ccd64fa0da None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:44.604134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307570c-7356-450d-ad1d-b4ccd64fa0da None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:44.605779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:44.638307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f8e3-185b-43f4-91ef-0f5081e599a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:44.638961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f8e3-185b-43f4-91ef-0f5081e599a4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:44.639236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f8e3-185b-43f4-91ef-0f5081e599a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:44.639505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f8e3-185b-43f4-91ef-0f5081e599a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:44.646599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4f8e3-185b-43f4-91ef-0f5081e599a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:44.648018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4f8e3-185b-43f4-91ef-0f5081e599a4 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-f9d4f8e3-185b-43f4-91ef-0f5081e599a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:36:44.661386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:44.662542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:44.663702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 605/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:44.669103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec74a9-745b-4541-b19c-2cb6a290849f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:44.769127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec74a9-745b-4541-b19c-2cb6a290849f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:44.803094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f76e6d-994a-4ff7-b7ce-2b59bd534908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:44.803694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f76e6d-994a-4ff7-b7ce-2b59bd534908 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:44.803974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f76e6d-994a-4ff7-b7ce-2b59bd534908 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:44.804244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f76e6d-994a-4ff7-b7ce-2b59bd534908 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:44.804958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f76e6d-994a-4ff7-b7ce-2b59bd534908 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:44.805294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f76e6d-994a-4ff7-b7ce-2b59bd534908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:44.806745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f76e6d-994a-4ff7-b7ce-2b59bd534908 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:44.824113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 608/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 18:36:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:44.830033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f8e3-185b-43f4-91ef-0f5081e599a4 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:44.930448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f8e3-185b-43f4-91ef-0f5081e599a4 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:44.965393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7d33b-ba5d-4dd9-af40-b4518c24808c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:44.965989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7d33b-ba5d-4dd9-af40-b4518c24808c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:44.966268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7d33b-ba5d-4dd9-af40-b4518c24808c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:44.966538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7d33b-ba5d-4dd9-af40-b4518c24808c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:44.996751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:45.035833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7d33b-ba5d-4dd9-af40-b4518c24808c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:45.037314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7d33b-ba5d-4dd9-af40-b4518c24808c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:45.135551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 606/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:36:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:36:45.141533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f76e6d-994a-4ff7-b7ce-2b59bd534908 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:45.161820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:45.242115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f76e6d-994a-4ff7-b7ce-2b59bd534908 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:45.277812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d536e26-bd2c-4442-a772-b9004fd52361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:45.278468 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d536e26-bd2c-4442-a772-b9004fd52361 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:45.278754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d536e26-bd2c-4442-a772-b9004fd52361 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:45.279023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d536e26-bd2c-4442-a772-b9004fd52361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:45.288966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d536e26-bd2c-4442-a772-b9004fd52361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:45.290460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d536e26-bd2c-4442-a772-b9004fd52361 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-2d536e26-bd2c-4442-a772-b9004fd52361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:36:45.305030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:45.309464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:36:45.310602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:36:45.310602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:45.310602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:36:45.310602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 609/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 18:36:45.312523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7d33b-ba5d-4dd9-af40-b4518c24808c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:45.376688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:45.414660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7d33b-ba5d-4dd9-af40-b4518c24808c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:45.449083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed811564-21b1-4c58-9ab0-ab718d96e58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:45.449687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed811564-21b1-4c58-9ab0-ab718d96e58a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:45.449973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed811564-21b1-4c58-9ab0-ab718d96e58a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:45.450251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed811564-21b1-4c58-9ab0-ab718d96e58a None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:45.450885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed811564-21b1-4c58-9ab0-ab718d96e58a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:45.451188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed811564-21b1-4c58-9ab0-ab718d96e58a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:45.452543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed811564-21b1-4c58-9ab0-ab718d96e58a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:45.464686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 607/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:45.470827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d536e26-bd2c-4442-a772-b9004fd52361 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:45.541464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:45.572015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d536e26-bd2c-4442-a772-b9004fd52361 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:45.607565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40173ce2-1e48-4c78-91b8-258222fea522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:45.608199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40173ce2-1e48-4c78-91b8-258222fea522 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:45.608478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40173ce2-1e48-4c78-91b8-258222fea522 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:45.608951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40173ce2-1e48-4c78-91b8-258222fea522 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:45.615711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40173ce2-1e48-4c78-91b8-258222fea522 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:45.617186 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40173ce2-1e48-4c78-91b8-258222fea522 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-40173ce2-1e48-4c78-91b8-258222fea522 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:36:45.632824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:45.634447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:45.635636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:36:45.635636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:36:45.635636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:36:45.635636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:36:45.635636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:36:45.635636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:36:45.635636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:36:45.635636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:36:45.635636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 610/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:45.640676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed811564-21b1-4c58-9ab0-ab718d96e58a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:45.741237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed811564-21b1-4c58-9ab0-ab718d96e58a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:45.765283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:45.777445 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d869403-83c1-4503-9a45-20b64fc96c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:45.778046 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d869403-83c1-4503-9a45-20b64fc96c12 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:45.778329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d869403-83c1-4503-9a45-20b64fc96c12 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:45.778606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d869403-83c1-4503-9a45-20b64fc96c12 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:45.779297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d869403-83c1-4503-9a45-20b64fc96c12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:45.779613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d869403-83c1-4503-9a45-20b64fc96c12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:45.781072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d869403-83c1-4503-9a45-20b64fc96c12 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:45.792652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 608/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:36:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:45.798813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40173ce2-1e48-4c78-91b8-258222fea522 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:45.898324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40173ce2-1e48-4c78-91b8-258222fea522 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:45.926444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:45.932907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804c507-67b0-4c88-aa97-5c7b6edc837e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:45.933533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804c507-67b0-4c88-aa97-5c7b6edc837e None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:45.933808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804c507-67b0-4c88-aa97-5c7b6edc837e None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:45.934079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804c507-67b0-4c88-aa97-5c7b6edc837e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:45.934786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804c507-67b0-4c88-aa97-5c7b6edc837e None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=a074714e365648b88c9d3a0495a17b2a) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:45.935090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804c507-67b0-4c88-aa97-5c7b6edc837e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:45.936577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804c507-67b0-4c88-aa97-5c7b6edc837e None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:45.941808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 611/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:36:45 2022] GET /identity/v3/role_assignments?user.id=a074714e365648b88c9d3a0495a17b2a&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:48.150791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a2fcc-3f76-4a19-b048-b9f8dbfcf012 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:48.151347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a2fcc-3f76-4a19-b048-b9f8dbfcf012 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:48.151347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a2fcc-3f76-4a19-b048-b9f8dbfcf012 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:48.152706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 609/1220] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:36:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:36:48.157959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116de646-aad9-41d8-a548-12cef7521715 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:48.158377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116de646-aad9-41d8-a548-12cef7521715 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:48.158516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116de646-aad9-41d8-a548-12cef7521715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:48.177232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-116de646-aad9-41d8-a548-12cef7521715 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:36:48.178415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-116de646-aad9-41d8-a548-12cef7521715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:36:48.246036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 612/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:36:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:36:48.513129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a2fcc-3f76-4a19-b048-b9f8dbfcf012 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:48.616111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a2fcc-3f76-4a19-b048-b9f8dbfcf012 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:48.650769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d590c-d692-40c7-9c98-bc7e15317947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:48.651396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d590c-d692-40c7-9c98-bc7e15317947 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:48.651677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d590c-d692-40c7-9c98-bc7e15317947 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:48.651951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d590c-d692-40c7-9c98-bc7e15317947 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:48.662085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d590c-d692-40c7-9c98-bc7e15317947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-6c1d590c-d692-40c7-9c98-bc7e15317947 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:36:48.666880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:48.670948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:48.671806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 610/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:36:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:48.674145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116de646-aad9-41d8-a548-12cef7521715 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:48.773167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116de646-aad9-41d8-a548-12cef7521715 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:48.796492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:48.806969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3460111-ce69-4710-ae88-f6ac9528fd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:48.807574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460111-ce69-4710-ae88-f6ac9528fd51 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:48.807858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460111-ce69-4710-ae88-f6ac9528fd51 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:48.808245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460111-ce69-4710-ae88-f6ac9528fd51 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:48.808910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3460111-ce69-4710-ae88-f6ac9528fd51 None admin] RBAC: Adding query filter params (name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:48.809251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3460111-ce69-4710-ae88-f6ac9528fd51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:48.810610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3460111-ce69-4710-ae88-f6ac9528fd51 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:48.818337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 613/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:36:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:48.824767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d590c-d692-40c7-9c98-bc7e15317947 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:48.868814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:48.923716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d590c-d692-40c7-9c98-bc7e15317947 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:48.957762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e3eb5-f49d-4fbb-b3f2-3d8d3960f8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:48.958384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e3eb5-f49d-4fbb-b3f2-3d8d3960f8fd None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:48.958699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e3eb5-f49d-4fbb-b3f2-3d8d3960f8fd None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:48.958936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e3eb5-f49d-4fbb-b3f2-3d8d3960f8fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:48.969268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e3eb5-f49d-4fbb-b3f2-3d8d3960f8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:48.970976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e3eb5-f49d-4fbb-b3f2-3d8d3960f8fd None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-a35e3eb5-f49d-4fbb-b3f2-3d8d3960f8fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:36:48.986178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:48.987290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:36:48.988378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:36:48.988378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:48.988378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:48.988378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 611/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:48.993705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3460111-ce69-4710-ae88-f6ac9528fd51 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:49.094201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3460111-ce69-4710-ae88-f6ac9528fd51 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:49.127407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc259b8-4dfb-43bc-a1d3-2f967fb548a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:49.128010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc259b8-4dfb-43bc-a1d3-2f967fb548a1 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:49.128290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc259b8-4dfb-43bc-a1d3-2f967fb548a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:49.128565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc259b8-4dfb-43bc-a1d3-2f967fb548a1 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:49.129229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc259b8-4dfb-43bc-a1d3-2f967fb548a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:49.129539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc259b8-4dfb-43bc-a1d3-2f967fb548a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:49.130921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc259b8-4dfb-43bc-a1d3-2f967fb548a1 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:49.143124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 614/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:49.148845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e3eb5-f49d-4fbb-b3f2-3d8d3960f8fd None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:49.242180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:49.248658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e3eb5-f49d-4fbb-b3f2-3d8d3960f8fd None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:49.281869 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376302db-daec-4b09-a07f-aab1a6effe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:49.282484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376302db-daec-4b09-a07f-aab1a6effe59 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:49.282760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376302db-daec-4b09-a07f-aab1a6effe59 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:49.283030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376302db-daec-4b09-a07f-aab1a6effe59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:49.290783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376302db-daec-4b09-a07f-aab1a6effe59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:49.292295 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376302db-daec-4b09-a07f-aab1a6effe59 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-376302db-daec-4b09-a07f-aab1a6effe59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:36:49.303472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:49.304588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:49.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 612/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:49.310486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc259b8-4dfb-43bc-a1d3-2f967fb548a1 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:49.365366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:49.417864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc259b8-4dfb-43bc-a1d3-2f967fb548a1 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:49.454450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990b051-ee29-40bb-9122-a6c0da78eae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:49.455059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990b051-ee29-40bb-9122-a6c0da78eae8 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:49.455349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990b051-ee29-40bb-9122-a6c0da78eae8 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:49.455618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990b051-ee29-40bb-9122-a6c0da78eae8 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:49.456334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990b051-ee29-40bb-9122-a6c0da78eae8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:49.456642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990b051-ee29-40bb-9122-a6c0da78eae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:49.458089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990b051-ee29-40bb-9122-a6c0da78eae8 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:49.475980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 615/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 18:36:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:49.482029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376302db-daec-4b09-a07f-aab1a6effe59 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:49.580583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376302db-daec-4b09-a07f-aab1a6effe59 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:49.615785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8a9f2-37d5-45c1-89c6-3103a1194a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:49.616353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8a9f2-37d5-45c1-89c6-3103a1194a1f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:49.616628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8a9f2-37d5-45c1-89c6-3103a1194a1f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:49.616897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8a9f2-37d5-45c1-89c6-3103a1194a1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:49.628264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8a9f2-37d5-45c1-89c6-3103a1194a1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:49.629786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8a9f2-37d5-45c1-89c6-3103a1194a1f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-13f8a9f2-37d5-45c1-89c6-3103a1194a1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:36:49.644314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:49.648595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:36:49.650224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:36:49.650224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:49.650224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:49.650224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 613/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:49.651600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990b051-ee29-40bb-9122-a6c0da78eae8 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:49.756752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990b051-ee29-40bb-9122-a6c0da78eae8 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:49.759246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:49.790743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154f9ff-7d68-454c-a738-15ea30142c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:49.791334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154f9ff-7d68-454c-a738-15ea30142c92 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:49.791608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154f9ff-7d68-454c-a738-15ea30142c92 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:49.791874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154f9ff-7d68-454c-a738-15ea30142c92 None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:49.792518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154f9ff-7d68-454c-a738-15ea30142c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:49.792794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154f9ff-7d68-454c-a738-15ea30142c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:49.794202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154f9ff-7d68-454c-a738-15ea30142c92 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:49.804950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 616/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:49.810964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8a9f2-37d5-45c1-89c6-3103a1194a1f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:49.820470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:49.911389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8a9f2-37d5-45c1-89c6-3103a1194a1f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:49.945237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f933fe8-7fd0-47f7-8419-e7dc3c773e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:49.945863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f933fe8-7fd0-47f7-8419-e7dc3c773e7f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:49.946594 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f933fe8-7fd0-47f7-8419-e7dc3c773e7f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:49.946860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f933fe8-7fd0-47f7-8419-e7dc3c773e7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:49.953437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f933fe8-7fd0-47f7-8419-e7dc3c773e7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:49.954883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f933fe8-7fd0-47f7-8419-e7dc3c773e7f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-5f933fe8-7fd0-47f7-8419-e7dc3c773e7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:36:49.966920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:49.967971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:49.968942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:36:49.968942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:36:49.968942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:36:49.968942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:36:49.968942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:36:49.968942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:36:49.968942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:36:49.968942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:49.968942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 614/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:36:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:49.974155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154f9ff-7d68-454c-a738-15ea30142c92 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:50.078619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154f9ff-7d68-454c-a738-15ea30142c92 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:50.113199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67bb03-8838-4b15-b639-0548ac640530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:50.113801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67bb03-8838-4b15-b639-0548ac640530 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:50.114074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67bb03-8838-4b15-b639-0548ac640530 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:50.114342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67bb03-8838-4b15-b639-0548ac640530 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:50.115035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67bb03-8838-4b15-b639-0548ac640530 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:50.115358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67bb03-8838-4b15-b639-0548ac640530 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:50.116777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67bb03-8838-4b15-b639-0548ac640530 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:50.129411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 617/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:36:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:36:50.136064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f933fe8-7fd0-47f7-8419-e7dc3c773e7f None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:50.201798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:50.237861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f933fe8-7fd0-47f7-8419-e7dc3c773e7f None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:50.270753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345343d-79bf-40dc-98e1-be40e2d02e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:50.271400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345343d-79bf-40dc-98e1-be40e2d02e63 None admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:50.271677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345343d-79bf-40dc-98e1-be40e2d02e63 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:50.271948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345343d-79bf-40dc-98e1-be40e2d02e63 None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/a074714e365648b88c9d3a0495a17b2a/roles/72ffa7e7e6374ac395b22fe6f5723045` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:50.287113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345343d-79bf-40dc-98e1-be40e2d02e63 None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=a074714e365648b88c9d3a0495a17b2a, role_id=72ffa7e7e6374ac395b22fe6f5723045)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:50.288552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345343d-79bf-40dc-98e1-be40e2d02e63 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:50.348300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 615/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:36:50 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/a074714e365648b88c9d3a0495a17b2a/roles/72ffa7e7e6374ac395b22fe6f5723045 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:36:52.521648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8acea-174b-41e6-812d-4d1d154a47f3 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:52.522317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8acea-174b-41e6-812d-4d1d154a47f3 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:52.522317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8acea-174b-41e6-812d-4d1d154a47f3 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:52.523653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 618/1233] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:36:52.530537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a040b-595e-4d4f-a8de-757cb31f2fd2 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:52.530986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a040b-595e-4d4f-a8de-757cb31f2fd2 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:52.531096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a040b-595e-4d4f-a8de-757cb31f2fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:52.548677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-269a040b-595e-4d4f-a8de-757cb31f2fd2 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:36:52.549886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-269a040b-595e-4d4f-a8de-757cb31f2fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:36:52.615402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 616/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:36:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:36:52.892609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8acea-174b-41e6-812d-4d1d154a47f3 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:52.920155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:52.995829 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8acea-174b-41e6-812d-4d1d154a47f3 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:53.030096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1bb0c-6283-470c-b0db-fb2102773828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:53.030730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1bb0c-6283-470c-b0db-fb2102773828 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:53.031037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1bb0c-6283-470c-b0db-fb2102773828 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:53.031308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1bb0c-6283-470c-b0db-fb2102773828 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:53.042387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1bb0c-6283-470c-b0db-fb2102773828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-52d1bb0c-6283-470c-b0db-fb2102773828 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:36:53.047285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:53.048426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:36:53.049676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 619/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 8 18:36:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:53.054452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a040b-595e-4d4f-a8de-757cb31f2fd2 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:53.102504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:53.155147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a040b-595e-4d4f-a8de-757cb31f2fd2 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:53.189143 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03df59-0c1f-4f4e-b04a-24c9405e9274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:53.189749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03df59-0c1f-4f4e-b04a-24c9405e9274 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:53.190040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03df59-0c1f-4f4e-b04a-24c9405e9274 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:53.190307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03df59-0c1f-4f4e-b04a-24c9405e9274 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:53.190966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03df59-0c1f-4f4e-b04a-24c9405e9274 None admin] RBAC: Adding query filter params (name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:53.191270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03df59-0c1f-4f4e-b04a-24c9405e9274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:53.192666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03df59-0c1f-4f4e-b04a-24c9405e9274 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:53.207781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 617/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 8 18:36:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:53.214651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1bb0c-6283-470c-b0db-fb2102773828 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:53.301280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:53.316355 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1bb0c-6283-470c-b0db-fb2102773828 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:53.349919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ac227-d776-4371-8db0-0818ad458d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:53.350597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac227-d776-4371-8db0-0818ad458d80 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:53.350873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac227-d776-4371-8db0-0818ad458d80 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:53.351142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ac227-d776-4371-8db0-0818ad458d80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:53.367804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ac227-d776-4371-8db0-0818ad458d80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:53.369293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ac227-d776-4371-8db0-0818ad458d80 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-f29ac227-d776-4371-8db0-0818ad458d80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:36:53.384125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:53.388586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:36:53.389775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:36:53.389775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:53.389775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:36:53.389775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 620/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:53.391284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03df59-0c1f-4f4e-b04a-24c9405e9274 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:53.496725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03df59-0c1f-4f4e-b04a-24c9405e9274 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:53.505637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:53.530390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424b345-1dc8-45fc-934f-b57322f9a967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:53.530986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424b345-1dc8-45fc-934f-b57322f9a967 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:53.531531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424b345-1dc8-45fc-934f-b57322f9a967 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:53.531531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424b345-1dc8-45fc-934f-b57322f9a967 None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:53.532129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424b345-1dc8-45fc-934f-b57322f9a967 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:53.532437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424b345-1dc8-45fc-934f-b57322f9a967 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:53.534079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424b345-1dc8-45fc-934f-b57322f9a967 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:53.545459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 618/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:53.551253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ac227-d776-4371-8db0-0818ad458d80 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:53.655362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ac227-d776-4371-8db0-0818ad458d80 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:53.689582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49d50b-4184-40cd-b956-853255716181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:53.690248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49d50b-4184-40cd-b956-853255716181 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:53.690530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49d50b-4184-40cd-b956-853255716181 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:53.690805 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49d50b-4184-40cd-b956-853255716181 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:53.693511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:53.699041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49d50b-4184-40cd-b956-853255716181 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:53.700495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49d50b-4184-40cd-b956-853255716181 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-5f49d50b-4184-40cd-b956-853255716181 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:36:53.711799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:53.715521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:36:53.716495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 621/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:53.719049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424b345-1dc8-45fc-934f-b57322f9a967 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:53.819275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424b345-1dc8-45fc-934f-b57322f9a967 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:53.855900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c1078-8a7f-4c5e-b728-64a5d6a68a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:53.856563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c1078-8a7f-4c5e-b728-64a5d6a68a17 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:53.856847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c1078-8a7f-4c5e-b728-64a5d6a68a17 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:53.857151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c1078-8a7f-4c5e-b728-64a5d6a68a17 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:53.857870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c1078-8a7f-4c5e-b728-64a5d6a68a17 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:53.858181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c1078-8a7f-4c5e-b728-64a5d6a68a17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:53.859598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c1078-8a7f-4c5e-b728-64a5d6a68a17 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:53.877529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 619/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 18:36:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:36:53.883383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49d50b-4184-40cd-b956-853255716181 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:53.987911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49d50b-4184-40cd-b956-853255716181 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:54.021955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbe7ad-19c8-4aa7-8abe-79c343bd4ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:54.022556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbe7ad-19c8-4aa7-8abe-79c343bd4ed7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:54.022838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbe7ad-19c8-4aa7-8abe-79c343bd4ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:54.023118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbe7ad-19c8-4aa7-8abe-79c343bd4ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:54.087657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:54.094220 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbe7ad-19c8-4aa7-8abe-79c343bd4ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:54.095920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbe7ad-19c8-4aa7-8abe-79c343bd4ed7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:54.193158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 622/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:36:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:36:54.199310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c1078-8a7f-4c5e-b728-64a5d6a68a17 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:54.227729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:54.297941 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c1078-8a7f-4c5e-b728-64a5d6a68a17 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:54.330549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7790b-d8d2-4242-96e4-210fdf3be667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:54.331176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7790b-d8d2-4242-96e4-210fdf3be667 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:54.331458 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7790b-d8d2-4242-96e4-210fdf3be667 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:54.331733 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7790b-d8d2-4242-96e4-210fdf3be667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:54.342571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7790b-d8d2-4242-96e4-210fdf3be667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:54.343953 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7790b-d8d2-4242-96e4-210fdf3be667 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-2fd7790b-d8d2-4242-96e4-210fdf3be667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:36:54.360290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:54.366214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:36:54.367352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:36:54.367352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:54.367352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:54.367352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 620/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:54.368968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbe7ad-19c8-4aa7-8abe-79c343bd4ed7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:54.459412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:54.465770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbe7ad-19c8-4aa7-8abe-79c343bd4ed7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:54.500038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bea2b-8c26-49f1-a7cc-bdcaaaa14ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:54.500676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bea2b-8c26-49f1-a7cc-bdcaaaa14ade None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:54.500922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bea2b-8c26-49f1-a7cc-bdcaaaa14ade None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:54.501237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bea2b-8c26-49f1-a7cc-bdcaaaa14ade None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:54.501874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bea2b-8c26-49f1-a7cc-bdcaaaa14ade None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:54.502185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bea2b-8c26-49f1-a7cc-bdcaaaa14ade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:54.503531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bea2b-8c26-49f1-a7cc-bdcaaaa14ade None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:54.514551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 623/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:54.520526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7790b-d8d2-4242-96e4-210fdf3be667 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:54.600223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:54.623138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7790b-d8d2-4242-96e4-210fdf3be667 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:54.658169 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c093d2-69bb-46f5-b3b9-dc63a6f7c46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:54.658796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c093d2-69bb-46f5-b3b9-dc63a6f7c46e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:54.659078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c093d2-69bb-46f5-b3b9-dc63a6f7c46e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:54.659353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c093d2-69bb-46f5-b3b9-dc63a6f7c46e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:54.665971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c093d2-69bb-46f5-b3b9-dc63a6f7c46e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:54.667582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c093d2-69bb-46f5-b3b9-dc63a6f7c46e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-c0c093d2-69bb-46f5-b3b9-dc63a6f7c46e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:36:54.680575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:54.681778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:54.682852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:36:54.682852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:36:54.682852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:36:54.682852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:36:54.682852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:36:54.682852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:36:54.682852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:36:54.682852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:54.682852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 621/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:36:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:54.687710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bea2b-8c26-49f1-a7cc-bdcaaaa14ade None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:54.787709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bea2b-8c26-49f1-a7cc-bdcaaaa14ade None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:54.820064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9382e-758e-4a1a-b17e-c4cdb05cf4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:54.820660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9382e-758e-4a1a-b17e-c4cdb05cf4d0 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:54.821577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9382e-758e-4a1a-b17e-c4cdb05cf4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:54.821854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9382e-758e-4a1a-b17e-c4cdb05cf4d0 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:54.822530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9382e-758e-4a1a-b17e-c4cdb05cf4d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:54.822834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9382e-758e-4a1a-b17e-c4cdb05cf4d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:54.824244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9382e-758e-4a1a-b17e-c4cdb05cf4d0 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:54.835766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 624/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:36:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:54.842529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c093d2-69bb-46f5-b3b9-dc63a6f7c46e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:54.943129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c093d2-69bb-46f5-b3b9-dc63a6f7c46e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:54.976623 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab8113-3b6a-4dc5-8008-1e7f45b01957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:54.977252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8113-3b6a-4dc5-8008-1e7f45b01957 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:54.977529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8113-3b6a-4dc5-8008-1e7f45b01957 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:54.977790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8113-3b6a-4dc5-8008-1e7f45b01957 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:54.978504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab8113-3b6a-4dc5-8008-1e7f45b01957 None admin] RBAC: Adding query filter params (role.id=72ffa7e7e6374ac395b22fe6f5723045, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=a074714e365648b88c9d3a0495a17b2a) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:54.978806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab8113-3b6a-4dc5-8008-1e7f45b01957 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:54.980218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab8113-3b6a-4dc5-8008-1e7f45b01957 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:54.985887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 622/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:36:54 2022] GET /identity/v3/role_assignments?user.id=a074714e365648b88c9d3a0495a17b2a&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=72ffa7e7e6374ac395b22fe6f5723045 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:57.220436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd633-98b5-4801-b7e5-cda11f0fd4b6 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:57.221340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd633-98b5-4801-b7e5-cda11f0fd4b6 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:57.221340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd633-98b5-4801-b7e5-cda11f0fd4b6 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:57.222620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 625/1247] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:36:57.228267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01869f-4f80-4c14-bb5b-282b0236fbe1 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:57.228719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01869f-4f80-4c14-bb5b-282b0236fbe1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:57.228830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01869f-4f80-4c14-bb5b-282b0236fbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:57.235148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:57.247852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-9e01869f-4f80-4c14-bb5b-282b0236fbe1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:36:57.249062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9e01869f-4f80-4c14-bb5b-282b0236fbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:36:57.316419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 623/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:36:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:36:57.589637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fd633-98b5-4801-b7e5-cda11f0fd4b6 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:57.610864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:57.691356 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fd633-98b5-4801-b7e5-cda11f0fd4b6 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:57.725209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f6220-4868-4fb0-ac82-a60e53415aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:57.725839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f6220-4868-4fb0-ac82-a60e53415aaf None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:57.726121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f6220-4868-4fb0-ac82-a60e53415aaf None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:57.726395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f6220-4868-4fb0-ac82-a60e53415aaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:57.736485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f6220-4868-4fb0-ac82-a60e53415aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-422f6220-4868-4fb0-ac82-a60e53415aaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:36:57.741624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:57.742769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:36:57.743738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 626/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:36:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:57.749926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01869f-4f80-4c14-bb5b-282b0236fbe1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:57.849467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01869f-4f80-4c14-bb5b-282b0236fbe1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:57.878351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:57.883174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3481d2-f421-4718-96a8-db84381462a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:57.884663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3481d2-f421-4718-96a8-db84381462a0 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:57.884942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3481d2-f421-4718-96a8-db84381462a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:57.885236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3481d2-f421-4718-96a8-db84381462a0 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:57.885935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3481d2-f421-4718-96a8-db84381462a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:57.886233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3481d2-f421-4718-96a8-db84381462a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:57.887589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3481d2-f421-4718-96a8-db84381462a0 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:57.895545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 624/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:36:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:57.902054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f6220-4868-4fb0-ac82-a60e53415aaf None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:57.979266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:58.001667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f6220-4868-4fb0-ac82-a60e53415aaf None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:58.034027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c942c-e92f-40d5-905c-119d4a897fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:58.034644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c942c-e92f-40d5-905c-119d4a897fd9 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:58.034919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c942c-e92f-40d5-905c-119d4a897fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:58.035195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c942c-e92f-40d5-905c-119d4a897fd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:58.045439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c942c-e92f-40d5-905c-119d4a897fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:58.046860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c942c-e92f-40d5-905c-119d4a897fd9 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-e31c942c-e92f-40d5-905c-119d4a897fd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:36:58.060947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:58.067821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:36:58.069016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:36:58.069016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:58.069016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:36:58.069016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 627/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:58.069343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3481d2-f421-4718-96a8-db84381462a0 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:58.167482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3481d2-f421-4718-96a8-db84381462a0 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:58.200572 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa0c19-935b-4ef4-9aa3-7d4669151d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:58.201202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa0c19-935b-4ef4-9aa3-7d4669151d2c None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:58.201492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa0c19-935b-4ef4-9aa3-7d4669151d2c None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:58.201765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa0c19-935b-4ef4-9aa3-7d4669151d2c None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:58.202398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa0c19-935b-4ef4-9aa3-7d4669151d2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:58.202702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa0c19-935b-4ef4-9aa3-7d4669151d2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:58.204074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa0c19-935b-4ef4-9aa3-7d4669151d2c None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:58.215868 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 625/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:58.221918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c942c-e92f-40d5-905c-119d4a897fd9 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:58.322179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c942c-e92f-40d5-905c-119d4a897fd9 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:58.349143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:58.357076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7da87e-90c2-4399-8065-23923b716906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:58.357709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7da87e-90c2-4399-8065-23923b716906 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:58.358038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7da87e-90c2-4399-8065-23923b716906 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:58.358313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7da87e-90c2-4399-8065-23923b716906 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:58.365733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7da87e-90c2-4399-8065-23923b716906 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:58.367155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7da87e-90c2-4399-8065-23923b716906 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-ce7da87e-90c2-4399-8065-23923b716906 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:36:58.378173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:58.379260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:36:58.380596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 628/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:58.385800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa0c19-935b-4ef4-9aa3-7d4669151d2c None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:58.440689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:58.486204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa0c19-935b-4ef4-9aa3-7d4669151d2c None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:58.520126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6ec6b-4d09-4bf7-aed4-f00e87932b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:58.520806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6ec6b-4d09-4bf7-aed4-f00e87932b52 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:58.521121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6ec6b-4d09-4bf7-aed4-f00e87932b52 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:58.521399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6ec6b-4d09-4bf7-aed4-f00e87932b52 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:58.522115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6ec6b-4d09-4bf7-aed4-f00e87932b52 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:58.522423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6ec6b-4d09-4bf7-aed4-f00e87932b52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:58.523843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6ec6b-4d09-4bf7-aed4-f00e87932b52 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:58.542630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 626/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 18:36:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:58.548427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7da87e-90c2-4399-8065-23923b716906 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:58.650919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7da87e-90c2-4399-8065-23923b716906 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:58.684183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb106b-629c-4612-a7ec-cd0d19523c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:58.684787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb106b-629c-4612-a7ec-cd0d19523c0c None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:58.685090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb106b-629c-4612-a7ec-cd0d19523c0c None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:58.685368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb106b-629c-4612-a7ec-cd0d19523c0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:58.740249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:58.752037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb106b-629c-4612-a7ec-cd0d19523c0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:58.753509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb106b-629c-4612-a7ec-cd0d19523c0c None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:58.850931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 629/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:36:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:36:58.857553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6ec6b-4d09-4bf7-aed4-f00e87932b52 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:58.960569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6ec6b-4d09-4bf7-aed4-f00e87932b52 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:58.976982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:58.999449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32ea98-ac79-4255-9aae-292aa9b671a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:59.000022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32ea98-ac79-4255-9aae-292aa9b671a9 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:59.000283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32ea98-ac79-4255-9aae-292aa9b671a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:59.000603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32ea98-ac79-4255-9aae-292aa9b671a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:59.010897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32ea98-ac79-4255-9aae-292aa9b671a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:59.012339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32ea98-ac79-4255-9aae-292aa9b671a9 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-8a32ea98-ac79-4255-9aae-292aa9b671a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:36:59.027783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:59.028769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:36:59.029750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:36:59.029750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:36:59.029750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:59.029750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 627/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:59.034775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb106b-629c-4612-a7ec-cd0d19523c0c None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:59.126374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:59.138538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb106b-629c-4612-a7ec-cd0d19523c0c None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:59.171142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f4711-2b4c-47fd-a8d4-97a38fe1fcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:59.171766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4711-2b4c-47fd-a8d4-97a38fe1fcff None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:59.172043 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4711-2b4c-47fd-a8d4-97a38fe1fcff None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:59.172314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f4711-2b4c-47fd-a8d4-97a38fe1fcff None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:59.172937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f4711-2b4c-47fd-a8d4-97a38fe1fcff None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:59.173266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f4711-2b4c-47fd-a8d4-97a38fe1fcff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:59.174618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f4711-2b4c-47fd-a8d4-97a38fe1fcff None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:59.185848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 630/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:59.191721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32ea98-ac79-4255-9aae-292aa9b671a9 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:59.290938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32ea98-ac79-4255-9aae-292aa9b671a9 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:59.324370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8c732-4e33-4b74-9353-003da16a78cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:59.324998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c732-4e33-4b74-9353-003da16a78cd None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:59.325308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c732-4e33-4b74-9353-003da16a78cd None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:59.325584 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c732-4e33-4b74-9353-003da16a78cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:59.332395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8c732-4e33-4b74-9353-003da16a78cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:59.333867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8c732-4e33-4b74-9353-003da16a78cd None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-31a8c732-4e33-4b74-9353-003da16a78cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:36:59.346184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:36:59.347197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:36:59.348106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:36:59.348106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:36:59.348106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:36:59.348106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:36:59.348106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:36:59.348106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:36:59.348106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:36:59.348106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:36:59.348106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 628/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:36:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:36:59.353589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f4711-2b4c-47fd-a8d4-97a38fe1fcff None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:59.460319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f4711-2b4c-47fd-a8d4-97a38fe1fcff None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:59.494774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8ccf2-8244-41cf-9dcd-c7b566d32161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:59.495456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8ccf2-8244-41cf-9dcd-c7b566d32161 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:59.495729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8ccf2-8244-41cf-9dcd-c7b566d32161 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:59.495996 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8ccf2-8244-41cf-9dcd-c7b566d32161 None admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:59.496670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8ccf2-8244-41cf-9dcd-c7b566d32161 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:59.496972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8ccf2-8244-41cf-9dcd-c7b566d32161 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:59.498439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8ccf2-8244-41cf-9dcd-c7b566d32161 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:59.511500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 631/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:36:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:36:59.517703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8c732-4e33-4b74-9353-003da16a78cd None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:36:59.543418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:36:59.619071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8c732-4e33-4b74-9353-003da16a78cd None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:36:59.653980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd1d7d-dba9-4891-90ea-26f711b8da40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:36:59.654578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd1d7d-dba9-4891-90ea-26f711b8da40 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:36:59.654858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd1d7d-dba9-4891-90ea-26f711b8da40 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:36:59.655132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd1d7d-dba9-4891-90ea-26f711b8da40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:36:59.655851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd1d7d-dba9-4891-90ea-26f711b8da40 None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=a074714e365648b88c9d3a0495a17b2a) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:36:59.656159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd1d7d-dba9-4891-90ea-26f711b8da40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:36:59.657622 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd1d7d-dba9-4891-90ea-26f711b8da40 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:36:59.662951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 629/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:36:59 2022] GET /identity/v3/role_assignments?user.id=a074714e365648b88c9d3a0495a17b2a&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=5c777de40b8449329d714d8c2229115e => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:01.822645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721548ae-7ca9-463b-8579-98d4bddace49 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:01.823225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721548ae-7ca9-463b-8579-98d4bddace49 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:01.823225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721548ae-7ca9-463b-8579-98d4bddace49 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:01.824520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 632/1261] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:37:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:37:01.829666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40015171-0847-4d24-8c88-03519715fcfb None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:01.830077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40015171-0847-4d24-8c88-03519715fcfb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:01.830214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40015171-0847-4d24-8c88-03519715fcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:01.848322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-40015171-0847-4d24-8c88-03519715fcfb None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:37:01.849530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-40015171-0847-4d24-8c88-03519715fcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:37:01.923115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 630/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:37:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:37:02.204183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721548ae-7ca9-463b-8579-98d4bddace49 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:02.215273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:02.323101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721548ae-7ca9-463b-8579-98d4bddace49 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:02.356707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e4c64-b1a0-4c14-ab6d-8d1b6cdc1fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:02.357353 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e4c64-b1a0-4c14-ab6d-8d1b6cdc1fb7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:02.357632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e4c64-b1a0-4c14-ab6d-8d1b6cdc1fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:02.357902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e4c64-b1a0-4c14-ab6d-8d1b6cdc1fb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:02.368336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e4c64-b1a0-4c14-ab6d-8d1b6cdc1fb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-806e4c64-b1a0-4c14-ab6d-8d1b6cdc1fb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:37:02.373149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:02.374281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:37:02.375207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 633/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:37:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:02.380475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40015171-0847-4d24-8c88-03519715fcfb None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:02.398892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:02.481931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40015171-0847-4d24-8c88-03519715fcfb None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:02.515812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5bd53-32f0-4dd3-8fbc-2d5142f983d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:02.516412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5bd53-32f0-4dd3-8fbc-2d5142f983d4 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:02.516691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5bd53-32f0-4dd3-8fbc-2d5142f983d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:02.516962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5bd53-32f0-4dd3-8fbc-2d5142f983d4 None admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:02.517648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5bd53-32f0-4dd3-8fbc-2d5142f983d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:02.518000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5bd53-32f0-4dd3-8fbc-2d5142f983d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:02.519328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5bd53-32f0-4dd3-8fbc-2d5142f983d4 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:02.528039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 631/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:37:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:02.533967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e4c64-b1a0-4c14-ab6d-8d1b6cdc1fb7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:02.611993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:02.646090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e4c64-b1a0-4c14-ab6d-8d1b6cdc1fb7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:02.682499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599d22e-e9de-46d2-b11c-d02a7add8c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:02.683129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599d22e-e9de-46d2-b11c-d02a7add8c6b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:02.683418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599d22e-e9de-46d2-b11c-d02a7add8c6b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:02.683740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599d22e-e9de-46d2-b11c-d02a7add8c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:02.694193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599d22e-e9de-46d2-b11c-d02a7add8c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:02.695625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599d22e-e9de-46d2-b11c-d02a7add8c6b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-7599d22e-e9de-46d2-b11c-d02a7add8c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:37:02.710109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:02.711234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:37:02.712169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:37:02.712169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:37:02.712169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:37:02.712169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 634/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:02.717481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5bd53-32f0-4dd3-8fbc-2d5142f983d4 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:02.799955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:02.818399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5bd53-32f0-4dd3-8fbc-2d5142f983d4 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:02.853376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d9d3c-c542-4a15-9f93-0c5f4394d71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:02.853966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d9d3c-c542-4a15-9f93-0c5f4394d71e None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:02.854247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d9d3c-c542-4a15-9f93-0c5f4394d71e None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:02.854519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d9d3c-c542-4a15-9f93-0c5f4394d71e None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:02.855155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d9d3c-c542-4a15-9f93-0c5f4394d71e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:02.855459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d9d3c-c542-4a15-9f93-0c5f4394d71e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:02.856851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d9d3c-c542-4a15-9f93-0c5f4394d71e None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:02.868391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 632/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:02.874242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599d22e-e9de-46d2-b11c-d02a7add8c6b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:02.977905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599d22e-e9de-46d2-b11c-d02a7add8c6b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:02.997309 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:03.011432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7653a92-a72b-472c-a370-666d0d63f615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:03.012060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7653a92-a72b-472c-a370-666d0d63f615 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:03.012342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7653a92-a72b-472c-a370-666d0d63f615 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:03.012616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7653a92-a72b-472c-a370-666d0d63f615 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:03.019946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7653a92-a72b-472c-a370-666d0d63f615 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:03.021491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7653a92-a72b-472c-a370-666d0d63f615 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-b7653a92-a72b-472c-a370-666d0d63f615 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:37:03.033083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:03.034283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:37:03.035403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 635/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:37:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:03.040234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d9d3c-c542-4a15-9f93-0c5f4394d71e None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:03.140574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d9d3c-c542-4a15-9f93-0c5f4394d71e None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:03.176354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bd1c0-f92b-4f06-8bfc-d99738adbdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:03.176963 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd1c0-f92b-4f06-8bfc-d99738adbdc5 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:03.177276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd1c0-f92b-4f06-8bfc-d99738adbdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:03.177555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd1c0-f92b-4f06-8bfc-d99738adbdc5 None admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:03.178329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bd1c0-f92b-4f06-8bfc-d99738adbdc5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:03.178655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bd1c0-f92b-4f06-8bfc-d99738adbdc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:03.180626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bd1c0-f92b-4f06-8bfc-d99738adbdc5 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:03.190661 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:03.199363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 633/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 18:37:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:03.205301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7653a92-a72b-472c-a370-666d0d63f615 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:03.304149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7653a92-a72b-472c-a370-666d0d63f615 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:03.337790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1b1a4-46c3-4ddd-95b4-e9385e81dc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:03.338408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1b1a4-46c3-4ddd-95b4-e9385e81dc5a None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:03.338689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1b1a4-46c3-4ddd-95b4-e9385e81dc5a None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:03.338961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1b1a4-46c3-4ddd-95b4-e9385e81dc5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:03.349901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1b1a4-46c3-4ddd-95b4-e9385e81dc5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:03.351326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1b1a4-46c3-4ddd-95b4-e9385e81dc5a None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-73e1b1a4-46c3-4ddd-95b4-e9385e81dc5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:37:03.366412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:03.372376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:37:03.373586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:37:03.373586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:37:03.373586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:37:03.373586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 636/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:03.373868 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bd1c0-f92b-4f06-8bfc-d99738adbdc5 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:03.478806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bd1c0-f92b-4f06-8bfc-d99738adbdc5 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:03.512115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ce87e-cc3a-408a-9f1f-4e2728a7ec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:03.512668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ce87e-cc3a-408a-9f1f-4e2728a7ec9d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:03.512948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ce87e-cc3a-408a-9f1f-4e2728a7ec9d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:03.513246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ce87e-cc3a-408a-9f1f-4e2728a7ec9d None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:03.513882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ce87e-cc3a-408a-9f1f-4e2728a7ec9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:03.514185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ce87e-cc3a-408a-9f1f-4e2728a7ec9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:03.515552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ce87e-cc3a-408a-9f1f-4e2728a7ec9d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:03.527227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 634/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:37:03.533021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1b1a4-46c3-4ddd-95b4-e9385e81dc5a None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:03.564542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:03.632017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1b1a4-46c3-4ddd-95b4-e9385e81dc5a None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:03.667519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03aada3-4a36-4119-8324-91f77d576693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:03.668147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03aada3-4a36-4119-8324-91f77d576693 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:03.668429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03aada3-4a36-4119-8324-91f77d576693 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:03.668702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03aada3-4a36-4119-8324-91f77d576693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:03.675425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03aada3-4a36-4119-8324-91f77d576693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:03.676838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03aada3-4a36-4119-8324-91f77d576693 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-d03aada3-4a36-4119-8324-91f77d576693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:37:03.688729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:03.695049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:03.696582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:37:03.696582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:37:03.696582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:37:03.696582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:37:03.696582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:37:03.696582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:37:03.696582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:37:03.696582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:37:03.696582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 637/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:37:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:03.697130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ce87e-cc3a-408a-9f1f-4e2728a7ec9d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:03.756035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:03.797038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ce87e-cc3a-408a-9f1f-4e2728a7ec9d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:03.829316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88290c0-422f-4d87-b4a2-4b01a0bb6290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:03.829909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88290c0-422f-4d87-b4a2-4b01a0bb6290 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:03.830187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88290c0-422f-4d87-b4a2-4b01a0bb6290 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:03.830458 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88290c0-422f-4d87-b4a2-4b01a0bb6290 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:03.831136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88290c0-422f-4d87-b4a2-4b01a0bb6290 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:03.831440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88290c0-422f-4d87-b4a2-4b01a0bb6290 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:03.832846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88290c0-422f-4d87-b4a2-4b01a0bb6290 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:03.845971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 635/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:37:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:03.852116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03aada3-4a36-4119-8324-91f77d576693 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:03.944903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:03.953970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03aada3-4a36-4119-8324-91f77d576693 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:03.990369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5819f-8a72-4ab0-b4e1-304ad3d491e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:03.991022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5819f-8a72-4ab0-b4e1-304ad3d491e3 None admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:03.991299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5819f-8a72-4ab0-b4e1-304ad3d491e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:03.991568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5819f-8a72-4ab0-b4e1-304ad3d491e3 None admin] PATH_INFO: `/v3/projects/371cca2196e34bd181a09086679cb0e7/users/a074714e365648b88c9d3a0495a17b2a/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:03.994130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5819f-8a72-4ab0-b4e1-304ad3d491e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=371cca2196e34bd181a09086679cb0e7, user_id=a074714e365648b88c9d3a0495a17b2a, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:03.995582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5819f-8a72-4ab0-b4e1-304ad3d491e3 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:04.148583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 638/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 8 18:37:03 2022] PUT /identity/v3/projects/371cca2196e34bd181a09086679cb0e7/users/a074714e365648b88c9d3a0495a17b2a/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:37:06.318969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492d0dd-5012-461a-a3ad-2a5cd195577c None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:06.319523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492d0dd-5012-461a-a3ad-2a5cd195577c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:06.319620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492d0dd-5012-461a-a3ad-2a5cd195577c None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:06.320948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 636/1274] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:37:06.326313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76b714-c838-4fff-a313-a22be3551ca8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:06.326724 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76b714-c838-4fff-a313-a22be3551ca8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:06.326855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76b714-c838-4fff-a313-a22be3551ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:06.345453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-ad76b714-c838-4fff-a313-a22be3551ca8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:37:06.346945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ad76b714-c838-4fff-a313-a22be3551ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:37:06.412925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 639/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:37:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:37:06.691400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492d0dd-5012-461a-a3ad-2a5cd195577c None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:06.796213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492d0dd-5012-461a-a3ad-2a5cd195577c None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:06.814307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:06.831247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ae12d-dddc-49ee-b708-1bb663f324fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:06.831880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ae12d-dddc-49ee-b708-1bb663f324fe None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:06.832163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ae12d-dddc-49ee-b708-1bb663f324fe None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:06.832438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ae12d-dddc-49ee-b708-1bb663f324fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:06.844320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ae12d-dddc-49ee-b708-1bb663f324fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-f50ae12d-dddc-49ee-b708-1bb663f324fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 18:37:06.849409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:06.850706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:06.851605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 637/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 8 18:37:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:06.861877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76b714-c838-4fff-a313-a22be3551ca8 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:06.941499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:06.960211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76b714-c838-4fff-a313-a22be3551ca8 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:06.994664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8b401-6baa-4294-bf32-9c506c65e4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:06.995271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8b401-6baa-4294-bf32-9c506c65e4a7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:06.995555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8b401-6baa-4294-bf32-9c506c65e4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:06.995832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d8b401-6baa-4294-bf32-9c506c65e4a7 None admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:06.996494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8b401-6baa-4294-bf32-9c506c65e4a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:06.996799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8b401-6baa-4294-bf32-9c506c65e4a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:06.998267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d8b401-6baa-4294-bf32-9c506c65e4a7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:07.006332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 640/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 8 18:37:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:07.013084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ae12d-dddc-49ee-b708-1bb663f324fe None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:07.115302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ae12d-dddc-49ee-b708-1bb663f324fe None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:07.150949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521dfeb-dad9-457f-bd8f-e8cd6d6d628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:07.151572 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521dfeb-dad9-457f-bd8f-e8cd6d6d628a None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:07.151852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521dfeb-dad9-457f-bd8f-e8cd6d6d628a None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:07.152139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521dfeb-dad9-457f-bd8f-e8cd6d6d628a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:07.162215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521dfeb-dad9-457f-bd8f-e8cd6d6d628a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:07.163660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521dfeb-dad9-457f-bd8f-e8cd6d6d628a None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-d521dfeb-dad9-457f-bd8f-e8cd6d6d628a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:37:07.179899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:07.181599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:37:07.182698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:37:07.182698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:37:07.182698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:07.182698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 638/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:07.187897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8b401-6baa-4294-bf32-9c506c65e4a7 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:07.313348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d8b401-6baa-4294-bf32-9c506c65e4a7 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:07.350441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935d1d6-87d0-4d47-8b65-cabdd30ac8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:07.351154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d1d6-87d0-4d47-8b65-cabdd30ac8bf None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:07.351344 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d1d6-87d0-4d47-8b65-cabdd30ac8bf None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:07.351621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d1d6-87d0-4d47-8b65-cabdd30ac8bf None admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:07.353081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935d1d6-87d0-4d47-8b65-cabdd30ac8bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:07.353503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935d1d6-87d0-4d47-8b65-cabdd30ac8bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:07.354913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935d1d6-87d0-4d47-8b65-cabdd30ac8bf None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:07.378863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 641/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:07.379126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521dfeb-dad9-457f-bd8f-e8cd6d6d628a None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:07.415620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:07.494220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521dfeb-dad9-457f-bd8f-e8cd6d6d628a None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:07.531957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367809ac-e669-40c7-97f1-0c0cc2bf3adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:07.532608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367809ac-e669-40c7-97f1-0c0cc2bf3adb None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:07.532890 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367809ac-e669-40c7-97f1-0c0cc2bf3adb None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:07.533198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367809ac-e669-40c7-97f1-0c0cc2bf3adb None admin] PATH_INFO: `/v3/users/placement` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:07.541920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367809ac-e669-40c7-97f1-0c0cc2bf3adb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:07.543401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367809ac-e669-40c7-97f1-0c0cc2bf3adb None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-367809ac-e669-40c7-97f1-0c0cc2bf3adb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 18:37:07.556842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:07.558160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:07.559223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 639/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:37:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:07.564233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935d1d6-87d0-4d47-8b65-cabdd30ac8bf None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:07.569013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:07.677412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935d1d6-87d0-4d47-8b65-cabdd30ac8bf None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:07.715613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cae3d-08f4-4d58-a88a-e2dbb386a4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:07.716230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cae3d-08f4-4d58-a88a-e2dbb386a4a5 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:07.716509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cae3d-08f4-4d58-a88a-e2dbb386a4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:07.716780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cae3d-08f4-4d58-a88a-e2dbb386a4a5 None admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:07.717684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cae3d-08f4-4d58-a88a-e2dbb386a4a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:07.718072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cae3d-08f4-4d58-a88a-e2dbb386a4a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:07.719334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cae3d-08f4-4d58-a88a-e2dbb386a4a5 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:07.739591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 642/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 8 18:37:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:07.746586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367809ac-e669-40c7-97f1-0c0cc2bf3adb None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:07.837637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:07.847108 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367809ac-e669-40c7-97f1-0c0cc2bf3adb None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:07.879961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab3ea-746e-47bb-89fb-6792f87e6c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:07.880556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab3ea-746e-47bb-89fb-6792f87e6c29 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:07.880834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab3ea-746e-47bb-89fb-6792f87e6c29 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:07.881136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab3ea-746e-47bb-89fb-6792f87e6c29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:07.948722 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ab3ea-746e-47bb-89fb-6792f87e6c29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:07.950128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ab3ea-746e-47bb-89fb-6792f87e6c29 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:08.037736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:08.048374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 640/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 8 18:37:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 18:37:08.054528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cae3d-08f4-4d58-a88a-e2dbb386a4a5 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:08.117109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:08.155851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cae3d-08f4-4d58-a88a-e2dbb386a4a5 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:08.194403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7636886-e47d-4bc2-8f71-c02c72315818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:08.195031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7636886-e47d-4bc2-8f71-c02c72315818 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:08.195313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7636886-e47d-4bc2-8f71-c02c72315818 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:08.196099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7636886-e47d-4bc2-8f71-c02c72315818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:08.206300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7636886-e47d-4bc2-8f71-c02c72315818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:08.207674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7636886-e47d-4bc2-8f71-c02c72315818 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-f7636886-e47d-4bc2-8f71-c02c72315818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 18:37:08.223199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:08.224368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 18:37:08.225553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 18:37:08.225553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 18:37:08.225553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:37:08.225553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 643/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:08.230331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab3ea-746e-47bb-89fb-6792f87e6c29 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:08.326173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab3ea-746e-47bb-89fb-6792f87e6c29 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:08.359805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dcac4-37fc-45b1-86a9-09535cd273cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:08.366126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dcac4-37fc-45b1-86a9-09535cd273cd None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:08.366126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dcac4-37fc-45b1-86a9-09535cd273cd None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:08.366126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dcac4-37fc-45b1-86a9-09535cd273cd None admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:08.366126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dcac4-37fc-45b1-86a9-09535cd273cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:08.366126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dcac4-37fc-45b1-86a9-09535cd273cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:08.366126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620dcac4-37fc-45b1-86a9-09535cd273cd None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:08.374262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 641/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 8 18:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:08.380073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7636886-e47d-4bc2-8f71-c02c72315818 None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:08.479002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7636886-e47d-4bc2-8f71-c02c72315818 None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:08.493447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:08.513539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558354f7-3527-41a2-b882-49ac1f9cf89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:08.514160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558354f7-3527-41a2-b882-49ac1f9cf89f None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:08.514438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558354f7-3527-41a2-b882-49ac1f9cf89f None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:08.514707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558354f7-3527-41a2-b882-49ac1f9cf89f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:08.521117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558354f7-3527-41a2-b882-49ac1f9cf89f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:08.522535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558354f7-3527-41a2-b882-49ac1f9cf89f None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application [None req-558354f7-3527-41a2-b882-49ac1f9cf89f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 18:37:08.533428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:08.534491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:08.535748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 18:37:08.535748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 18:37:08.535748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 18:37:08.535748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 18:37:08.535748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 18:37:08.535748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 18:37:08.535748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 18:37:08.535748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: ERROR keystone.server.flask.application Sep 08 18:37:08.535748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 644/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 8 18:37:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:08.540681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dcac4-37fc-45b1-86a9-09535cd273cd None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:08.591337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:08.642150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dcac4-37fc-45b1-86a9-09535cd273cd None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:08.675227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c4c79-003f-48cf-94b8-061a3ed28127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:08.675833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4c79-003f-48cf-94b8-061a3ed28127 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:08.676118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4c79-003f-48cf-94b8-061a3ed28127 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:08.676393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4c79-003f-48cf-94b8-061a3ed28127 None admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:08.677105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c4c79-003f-48cf-94b8-061a3ed28127 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:08.677421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c4c79-003f-48cf-94b8-061a3ed28127 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:08.678884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c4c79-003f-48cf-94b8-061a3ed28127 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:08.691673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 642/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 8 18:37:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:37:08.698470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558354f7-3527-41a2-b882-49ac1f9cf89f None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:08.803588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558354f7-3527-41a2-b882-49ac1f9cf89f None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:08.838179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b96b90-6d5e-4e51-b14b-ceaadb360fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:08.838781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b96b90-6d5e-4e51-b14b-ceaadb360fb7 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:08.839064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b96b90-6d5e-4e51-b14b-ceaadb360fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:08.839342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b96b90-6d5e-4e51-b14b-ceaadb360fb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:08.840060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b96b90-6d5e-4e51-b14b-ceaadb360fb7 None admin] RBAC: Adding query filter params (role.id=5c777de40b8449329d714d8c2229115e, scope.project.id=371cca2196e34bd181a09086679cb0e7, user.id=a074714e365648b88c9d3a0495a17b2a) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:08.840373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b96b90-6d5e-4e51-b14b-ceaadb360fb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:08.841828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b96b90-6d5e-4e51-b14b-ceaadb360fb7 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:08.847844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 645/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 8 18:37:08 2022] GET /identity/v3/role_assignments?user.id=a074714e365648b88c9d3a0495a17b2a&scope.project.id=371cca2196e34bd181a09086679cb0e7&role.id=5c777de40b8449329d714d8c2229115e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:11.044817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fbdbb-c7ef-4ad5-8510-75bacbe8ef2a None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:11.045391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fbdbb-c7ef-4ad5-8510-75bacbe8ef2a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:11.045391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fbdbb-c7ef-4ad5-8510-75bacbe8ef2a None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:11.046771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 643/1288] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:37:11.051994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07b114-670c-4ff6-8e9a-9c725ce8c05f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:11.052425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07b114-670c-4ff6-8e9a-9c725ce8c05f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:11.052543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07b114-670c-4ff6-8e9a-9c725ce8c05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:11.071764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-bd07b114-670c-4ff6-8e9a-9c725ce8c05f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:37:11.073266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-bd07b114-670c-4ff6-8e9a-9c725ce8c05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:37:11.102683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:11.139815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 646/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:37:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:37:11.418243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fbdbb-c7ef-4ad5-8510-75bacbe8ef2a None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:11.529569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fbdbb-c7ef-4ad5-8510-75bacbe8ef2a None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:11.540575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:11.564780 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa922d5a-31dc-41b3-b5c3-7b8ec46e483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:11.565525 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa922d5a-31dc-41b3-b5c3-7b8ec46e483d None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:11.565877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa922d5a-31dc-41b3-b5c3-7b8ec46e483d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:11.566219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa922d5a-31dc-41b3-b5c3-7b8ec46e483d None admin] PATH_INFO: `/v3/services/placement` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:11.566806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa922d5a-31dc-41b3-b5c3-7b8ec46e483d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:11.568179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa922d5a-31dc-41b3-b5c3-7b8ec46e483d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-fa922d5a-31dc-41b3-b5c3-7b8ec46e483d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:11.577766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:11.578886 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:11.579889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 644/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 18:37:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:11.584870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07b114-670c-4ff6-8e9a-9c725ce8c05f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:11.685067 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07b114-670c-4ff6-8e9a-9c725ce8c05f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:11.718749 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae39013-ffde-490c-9e30-a89937e72371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:11.719350 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae39013-ffde-490c-9e30-a89937e72371 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:11.719626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae39013-ffde-490c-9e30-a89937e72371 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:11.719895 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae39013-ffde-490c-9e30-a89937e72371 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:11.720514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae39013-ffde-490c-9e30-a89937e72371 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:11.721135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae39013-ffde-490c-9e30-a89937e72371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:11.722434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae39013-ffde-490c-9e30-a89937e72371 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:11.727141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 647/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 8 18:37:11 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:11.732917 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa922d5a-31dc-41b3-b5c3-7b8ec46e483d None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:11.844607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa922d5a-31dc-41b3-b5c3-7b8ec46e483d None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:11.880919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d1125-85c7-4ab6-a061-1a8f6bcf1b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:11.881550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d1125-85c7-4ab6-a061-1a8f6bcf1b5f None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:11.881833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d1125-85c7-4ab6-a061-1a8f6bcf1b5f None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:11.882109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d1125-85c7-4ab6-a061-1a8f6bcf1b5f None admin] PATH_INFO: `/v3/services` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:11.882838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d1125-85c7-4ab6-a061-1a8f6bcf1b5f None admin] RBAC: Adding query filter params (type=placement) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:11.883213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d1125-85c7-4ab6-a061-1a8f6bcf1b5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:11.884739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d1125-85c7-4ab6-a061-1a8f6bcf1b5f None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:11.889278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 645/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 8 18:37:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:14.036653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61c25-a793-41f7-93bb-2b8b5e3723aa None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:14.037211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61c25-a793-41f7-93bb-2b8b5e3723aa None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:14.037317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f61c25-a793-41f7-93bb-2b8b5e3723aa None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:14.038665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 648/1293] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:37:14.044621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bcbb7-429b-4df2-bc8b-549a73d3defa None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:14.045093 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bcbb7-429b-4df2-bc8b-549a73d3defa None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:14.045211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bcbb7-429b-4df2-bc8b-549a73d3defa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:14.063373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-534bcbb7-429b-4df2-bc8b-549a73d3defa None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:37:14.064553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-534bcbb7-429b-4df2-bc8b-549a73d3defa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:37:14.094347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:14.130256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 646/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:37:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:37:14.396201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f61c25-a793-41f7-93bb-2b8b5e3723aa None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:14.426664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:14.501624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f61c25-a793-41f7-93bb-2b8b5e3723aa None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:14.535213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14ede8-16aa-4809-bc2a-29655533d9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:14.535818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14ede8-16aa-4809-bc2a-29655533d9c2 None admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:14.536097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14ede8-16aa-4809-bc2a-29655533d9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:14.536372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14ede8-16aa-4809-bc2a-29655533d9c2 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:14.537021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14ede8-16aa-4809-bc2a-29655533d9c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:14.538403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14ede8-16aa-4809-bc2a-29655533d9c2 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:14.628294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 649/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 8 18:37:14 2022] POST /identity/v3/services => generated 246 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:37:16.771057 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f3f00-90b7-43c3-88d4-bee073640996 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:16.771586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f3f00-90b7-43c3-88d4-bee073640996 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:16.771586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f3f00-90b7-43c3-88d4-bee073640996 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:16.772940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 647/1296] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:37:16.778773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d40ed-cb4e-4478-b328-67dc12713793 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:16.779168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d40ed-cb4e-4478-b328-67dc12713793 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:16.779322 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d40ed-cb4e-4478-b328-67dc12713793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:16.798258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-381d40ed-cb4e-4478-b328-67dc12713793 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:37:16.799449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-381d40ed-cb4e-4478-b328-67dc12713793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:37:16.883841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 650/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:37:16 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:37:17.149482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f3f00-90b7-43c3-88d4-bee073640996 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:17.264755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f3f00-90b7-43c3-88d4-bee073640996 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:17.299080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc347fa-01f6-4411-9e15-856cc244b7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:17.299693 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc347fa-01f6-4411-9e15-856cc244b7bc None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:17.299971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc347fa-01f6-4411-9e15-856cc244b7bc None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:17.300321 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc347fa-01f6-4411-9e15-856cc244b7bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:17.300897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc347fa-01f6-4411-9e15-856cc244b7bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:17.302272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc347fa-01f6-4411-9e15-856cc244b7bc None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-ccc347fa-01f6-4411-9e15-856cc244b7bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:17.311564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:17.312826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:17.313901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 648/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 18:37:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:17.318707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d40ed-cb4e-4478-b328-67dc12713793 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:17.337007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:17.417168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d40ed-cb4e-4478-b328-67dc12713793 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:17.450507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4af8b-3fa2-451c-9aa4-09f972767a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:17.451100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4af8b-3fa2-451c-9aa4-09f972767a1b None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:17.451376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4af8b-3fa2-451c-9aa4-09f972767a1b None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:17.451642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4af8b-3fa2-451c-9aa4-09f972767a1b None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:17.452254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4af8b-3fa2-451c-9aa4-09f972767a1b None admin] RBAC: Adding query filter params (name=placement) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:17.452552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4af8b-3fa2-451c-9aa4-09f972767a1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:17.454208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4af8b-3fa2-451c-9aa4-09f972767a1b None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:17.460035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 651/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 8 18:37:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:17.466660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc347fa-01f6-4411-9e15-856cc244b7bc None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:17.501953 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:17.575585 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc347fa-01f6-4411-9e15-856cc244b7bc None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:17.611334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2312a-209b-4807-aab5-3e72b09c6c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:17.612508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2312a-209b-4807-aab5-3e72b09c6c38 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:17.612840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2312a-209b-4807-aab5-3e72b09c6c38 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:17.613147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2312a-209b-4807-aab5-3e72b09c6c38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:17.613854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2312a-209b-4807-aab5-3e72b09c6c38 None admin] RBAC: Adding query filter params (interface=public, service_id=8baa4f3938c146faa7b14220cd90b47c, region_id=RegionOne) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:17.614164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2312a-209b-4807-aab5-3e72b09c6c38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:17.615566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2312a-209b-4807-aab5-3e72b09c6c38 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:17.620932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 649/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 8 18:37:17 2022] GET /identity/v3/endpoints?service_id=8baa4f3938c146faa7b14220cd90b47c&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:17.626633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4af8b-3fa2-451c-9aa4-09f972767a1b None admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:17.709711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:17.727796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4af8b-3fa2-451c-9aa4-09f972767a1b None admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:17.762240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84dfe3-31a1-4870-9dd4-e33f8ae165ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:17.762847 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84dfe3-31a1-4870-9dd4-e33f8ae165ce None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:17.763129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84dfe3-31a1-4870-9dd4-e33f8ae165ce None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:17.763404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84dfe3-31a1-4870-9dd4-e33f8ae165ce None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:17.764075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84dfe3-31a1-4870-9dd4-e33f8ae165ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:17.765472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84dfe3-31a1-4870-9dd4-e33f8ae165ce None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:17.772224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 652/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 8 18:37:17 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:37:19.955817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f3ed8-bb4b-4308-9bb7-391053b2c684 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:19.956339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f3ed8-bb4b-4308-9bb7-391053b2c684 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:19.956339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f3ed8-bb4b-4308-9bb7-391053b2c684 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:19.957851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 650/1302] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:37:19.963718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc918eb0-b2b6-4873-a5a1-ee2295f7d1b4 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:19.964142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc918eb0-b2b6-4873-a5a1-ee2295f7d1b4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:19.964298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc918eb0-b2b6-4873-a5a1-ee2295f7d1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:19.983224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-fc918eb0-b2b6-4873-a5a1-ee2295f7d1b4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:37:19.984409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-fc918eb0-b2b6-4873-a5a1-ee2295f7d1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:37:20.050779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 653/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:37:19 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 18:37:20.322671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f3ed8-bb4b-4308-9bb7-391053b2c684 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:20.416389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:20.423056 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f3ed8-bb4b-4308-9bb7-391053b2c684 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:20.456755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af77252-c6bb-494d-826b-29378ef42915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:20.457403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77252-c6bb-494d-826b-29378ef42915 None admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:20.457704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77252-c6bb-494d-826b-29378ef42915 None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:20.457976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af77252-c6bb-494d-826b-29378ef42915 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:20.458547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af77252-c6bb-494d-826b-29378ef42915 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:20.459889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af77252-c6bb-494d-826b-29378ef42915 None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application [None req-9af77252-c6bb-494d-826b-29378ef42915 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 18:37:20.468990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 18:37:20.470095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application with Lock( Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self._enter() Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application return self.creator() Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: ERROR keystone.server.flask.application Sep 08 18:37:20.471222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 651/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 8 18:37:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 18:37:20.476407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc918eb0-b2b6-4873-a5a1-ee2295f7d1b4 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:20.552712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:20.575098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc918eb0-b2b6-4873-a5a1-ee2295f7d1b4 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:20.613171 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf923f9-f94e-41ac-b70c-485319703c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:20.616740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf923f9-f94e-41ac-b70c-485319703c52 None admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:20.616740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf923f9-f94e-41ac-b70c-485319703c52 None admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:20.616740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf923f9-f94e-41ac-b70c-485319703c52 None admin] PATH_INFO: `/v3/services` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:20.616740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf923f9-f94e-41ac-b70c-485319703c52 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:37:20.616740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf923f9-f94e-41ac-b70c-485319703c52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:20.616740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf923f9-f94e-41ac-b70c-485319703c52 None admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:20.622632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 654/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 8 18:37:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:37:20.628429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af77252-c6bb-494d-826b-29378ef42915 None admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:37:20.731846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af77252-c6bb-494d-826b-29378ef42915 None admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:37:20.764313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23083c21-64c8-48ad-8fc4-40f649299e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:37:20.764924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23083c21-64c8-48ad-8fc4-40f649299e7d None admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:37:20.765215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23083c21-64c8-48ad-8fc4-40f649299e7d None admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:37:20.765489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23083c21-64c8-48ad-8fc4-40f649299e7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:37:20.766141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23083c21-64c8-48ad-8fc4-40f649299e7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:37:20.767506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23083c21-64c8-48ad-8fc4-40f649299e7d None admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:37:20.873860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 652/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:37:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:37:30.619290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=30 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:37:30.868994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:07.735614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27593ba-f533-41eb-82da-3ccdffc412ad None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:07.736155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27593ba-f533-41eb-82da-3ccdffc412ad None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:07.736155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27593ba-f533-41eb-82da-3ccdffc412ad None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:07.743451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf9902-e477-4c40-bc78-c7ba04649fcb None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:07.743933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf9902-e477-4c40-bc78-c7ba04649fcb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:07.744037 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf9902-e477-4c40-bc78-c7ba04649fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:07.747066 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 655/1307] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:07.827058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-5fcf9902-e477-4c40-bc78-c7ba04649fcb None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:07.828281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5fcf9902-e477-4c40-bc78-c7ba04649fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:07.938582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 653/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:43:08.239777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3f6b0-4fa7-4629-9cd8-0445d29a8815 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:08.240058 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3f6b0-4fa7-4629-9cd8-0445d29a8815 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:08.240596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3f6b0-4fa7-4629-9cd8-0445d29a8815 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:08.242243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 656/1309] 10.222.0.25 () {54 vars in 965 bytes} [Thu Sep 8 18:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:08.248104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946c70d-2ff4-479e-bb9c-08f17cf47f80 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:08.248400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946c70d-2ff4-479e-bb9c-08f17cf47f80 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:08.248679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946c70d-2ff4-479e-bb9c-08f17cf47f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:08.333388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c946c70d-2ff4-479e-bb9c-08f17cf47f80 None None] MFA Rules not processed for user `bcf30ef695154d3a97d7e589aca21233`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:08.334998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c946c70d-2ff4-479e-bb9c-08f17cf47f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:08.359689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:08.435116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 654/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Thu Sep 8 18:43:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:08.444622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3f6b0-4fa7-4629-9cd8-0445d29a8815 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:08.446695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-11a3f6b0-4fa7-4629-9cd8-0445d29a8815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:08.582487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3f6b0-4fa7-4629-9cd8-0445d29a8815 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:08.670614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a9f6d-2b51-4570-84aa-606a7d8e8c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:08.671230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a9f6d-2b51-4570-84aa-606a7d8e8c44 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:08.671509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a9f6d-2b51-4570-84aa-606a7d8e8c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:08.671780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a9f6d-2b51-4570-84aa-606a7d8e8c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:08.673400 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7f4a9f6d-2b51-4570-84aa-606a7d8e8c44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:08.732425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:08.789779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a9f6d-2b51-4570-84aa-606a7d8e8c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:08.791278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a9f6d-2b51-4570-84aa-606a7d8e8c44 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:08.811557 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 657/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:11.313927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9b6ff-3e21-4f90-af9d-9c7c23a906ad None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:11.314451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9b6ff-3e21-4f90-af9d-9c7c23a906ad None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:11.314544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9b6ff-3e21-4f90-af9d-9c7c23a906ad None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:11.315832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 655/1312] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:11.321955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191140a-7fca-4ee8-9216-03c6167e38e7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:11.322371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191140a-7fca-4ee8-9216-03c6167e38e7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:11.322511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191140a-7fca-4ee8-9216-03c6167e38e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:11.407694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-9191140a-7fca-4ee8-9216-03c6167e38e7 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:11.408904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9191140a-7fca-4ee8-9216-03c6167e38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:11.506419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 658/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:11.792061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9b6ff-3e21-4f90-af9d-9c7c23a906ad None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:11.793691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1eb9b6ff-3e21-4f90-af9d-9c7c23a906ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:11.904718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9b6ff-3e21-4f90-af9d-9c7c23a906ad None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:11.980137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:11.980398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79cb7e-aa79-4ea7-a786-b41a84ddc3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:11.981286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79cb7e-aa79-4ea7-a786-b41a84ddc3fb service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:11.981286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79cb7e-aa79-4ea7-a786-b41a84ddc3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:11.982437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79cb7e-aa79-4ea7-a786-b41a84ddc3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:11.983231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1a79cb7e-aa79-4ea7-a786-b41a84ddc3fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:12.100420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79cb7e-aa79-4ea7-a786-b41a84ddc3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:12.101892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79cb7e-aa79-4ea7-a786-b41a84ddc3fb service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:12.121072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 656/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:14.795281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cedea-2bbe-4d27-bfd7-8b0bcf09f38b None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:14.795835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cedea-2bbe-4d27-bfd7-8b0bcf09f38b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:14.795835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cedea-2bbe-4d27-bfd7-8b0bcf09f38b None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:14.797197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 659/1315] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:14.802361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940dabd9-3a4c-4941-a723-189a3dbaae33 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:14.803263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940dabd9-3a4c-4941-a723-189a3dbaae33 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:14.803596 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940dabd9-3a4c-4941-a723-189a3dbaae33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:14.869904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-940dabd9-3a4c-4941-a723-189a3dbaae33 None None] MFA Rules not processed for user `c6ad7503ff7243b2881542fdfef8df5c`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:14.871051 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-940dabd9-3a4c-4941-a723-189a3dbaae33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:14.923089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:14.967190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 657/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:15.256263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cedea-2bbe-4d27-bfd7-8b0bcf09f38b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:15.257824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-eb6cedea-2bbe-4d27-bfd7-8b0bcf09f38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:15.268420 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:15.366637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cedea-2bbe-4d27-bfd7-8b0bcf09f38b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:15.442025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e24d6e-1f6f-4d47-aeb8-55bf337fd69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:15.442625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24d6e-1f6f-4d47-aeb8-55bf337fd69e service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:15.442904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24d6e-1f6f-4d47-aeb8-55bf337fd69e service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:15.443179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24d6e-1f6f-4d47-aeb8-55bf337fd69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:15.445527 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e0e24d6e-1f6f-4d47-aeb8-55bf337fd69e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:15.509686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:15.558032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e24d6e-1f6f-4d47-aeb8-55bf337fd69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:15.559470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e24d6e-1f6f-4d47-aeb8-55bf337fd69e service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:15.577746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 660/1317] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:18.750539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efdb06-eced-4087-85e0-5fa9dccd8989 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:18.751164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efdb06-eced-4087-85e0-5fa9dccd8989 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:18.751164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efdb06-eced-4087-85e0-5fa9dccd8989 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:18.752476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 658/1318] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:18.758528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a7973-981c-46a2-b4c9-bced93a7f2d8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:18.758976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a7973-981c-46a2-b4c9-bced93a7f2d8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:18.759084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a7973-981c-46a2-b4c9-bced93a7f2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:18.826049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-dc7a7973-981c-46a2-b4c9-bced93a7f2d8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:18.827273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-dc7a7973-981c-46a2-b4c9-bced93a7f2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:18.924219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 661/1319] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:19.206646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efdb06-eced-4087-85e0-5fa9dccd8989 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:19.208188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c6efdb06-eced-4087-85e0-5fa9dccd8989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:19.319535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efdb06-eced-4087-85e0-5fa9dccd8989 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:19.394191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fcc34-6b98-42bf-8d99-0931244073d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:19.394790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fcc34-6b98-42bf-8d99-0931244073d6 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:19.395072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fcc34-6b98-42bf-8d99-0931244073d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:19.395349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fcc34-6b98-42bf-8d99-0931244073d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:19.397709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ed5fcc34-6b98-42bf-8d99-0931244073d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:19.404543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:19.505341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fcc34-6b98-42bf-8d99-0931244073d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:19.506970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fcc34-6b98-42bf-8d99-0931244073d6 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:19.524705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 659/1320] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:22.657242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d0209-1708-46f7-a7e2-9093efb48e83 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:22.657806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d0209-1708-46f7-a7e2-9093efb48e83 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:22.657806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d0209-1708-46f7-a7e2-9093efb48e83 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:22.659133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 662/1321] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:22.664456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed9087-f354-465a-882a-b4fa4f4c3767 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:22.664883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed9087-f354-465a-882a-b4fa4f4c3767 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:22.664994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed9087-f354-465a-882a-b4fa4f4c3767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:22.732496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-50ed9087-f354-465a-882a-b4fa4f4c3767 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:22.733740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-50ed9087-f354-465a-882a-b4fa4f4c3767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:22.792687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:22.837113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 660/1322] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:23.123520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d0209-1708-46f7-a7e2-9093efb48e83 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:23.125256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-cf5d0209-1708-46f7-a7e2-9093efb48e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:23.146870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:23.231934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d0209-1708-46f7-a7e2-9093efb48e83 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:23.311274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a168c-df48-4639-8c9c-07f2dd32bb07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:23.311894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a168c-df48-4639-8c9c-07f2dd32bb07 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:23.312179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a168c-df48-4639-8c9c-07f2dd32bb07 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:23.312455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a168c-df48-4639-8c9c-07f2dd32bb07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:23.314123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-da3a168c-df48-4639-8c9c-07f2dd32bb07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:23.389641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:23.427151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a168c-df48-4639-8c9c-07f2dd32bb07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:23.429367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3a168c-df48-4639-8c9c-07f2dd32bb07 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:23.447398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 663/1323] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:25.717177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10c78b-0d40-4a5f-a9d6-9324e8e930af None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:25.717726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10c78b-0d40-4a5f-a9d6-9324e8e930af None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:25.717726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10c78b-0d40-4a5f-a9d6-9324e8e930af None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:25.719128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 661/1324] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:25.725304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e9301-acb0-45c6-92c4-6a882dff1f65 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:25.725757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e9301-acb0-45c6-92c4-6a882dff1f65 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:25.725860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e9301-acb0-45c6-92c4-6a882dff1f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:25.793734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e03e9301-acb0-45c6-92c4-6a882dff1f65 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:25.794932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e03e9301-acb0-45c6-92c4-6a882dff1f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:25.898521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 664/1325] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:26.183314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10c78b-0d40-4a5f-a9d6-9324e8e930af None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:26.184932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ef10c78b-0d40-4a5f-a9d6-9324e8e930af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:26.292138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10c78b-0d40-4a5f-a9d6-9324e8e930af None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:26.369102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daf048-25bc-43e0-8982-098ae1032a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:26.369711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daf048-25bc-43e0-8982-098ae1032a98 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:26.369988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daf048-25bc-43e0-8982-098ae1032a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:26.370260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daf048-25bc-43e0-8982-098ae1032a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:26.371857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-81daf048-25bc-43e0-8982-098ae1032a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:26.389230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:26.483014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81daf048-25bc-43e0-8982-098ae1032a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:26.484724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81daf048-25bc-43e0-8982-098ae1032a98 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:26.503272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 662/1326] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:28.741669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f058fa-0654-4514-9629-cb34d3d00d7b None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:28.742223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f058fa-0654-4514-9629-cb34d3d00d7b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:28.742223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f058fa-0654-4514-9629-cb34d3d00d7b None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:28.743534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 665/1327] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:28.748915 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18da7bf-703c-47c8-a68b-cd3c35452209 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:28.749400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18da7bf-703c-47c8-a68b-cd3c35452209 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:28.749531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18da7bf-703c-47c8-a68b-cd3c35452209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:28.816370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e18da7bf-703c-47c8-a68b-cd3c35452209 None None] MFA Rules not processed for user `c6ad7503ff7243b2881542fdfef8df5c`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:28.817600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e18da7bf-703c-47c8-a68b-cd3c35452209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:28.883621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:28.916870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 663/1328] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:29.206990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f058fa-0654-4514-9629-cb34d3d00d7b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:29.208531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e1f058fa-0654-4514-9629-cb34d3d00d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:29.242837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:29.330080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f058fa-0654-4514-9629-cb34d3d00d7b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:29.417932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e126a9f-5a2e-4c05-9e21-6122e49d2b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:29.418547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e126a9f-5a2e-4c05-9e21-6122e49d2b4f service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:29.418825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e126a9f-5a2e-4c05-9e21-6122e49d2b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:29.419095 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e126a9f-5a2e-4c05-9e21-6122e49d2b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:29.420691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7e126a9f-5a2e-4c05-9e21-6122e49d2b4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:29.504737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:29.538084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e126a9f-5a2e-4c05-9e21-6122e49d2b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:29.539822 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e126a9f-5a2e-4c05-9e21-6122e49d2b4f service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:29.558906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 666/1329] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:32.300051 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cc94f-8843-4845-818b-5fb601a50820 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:32.300616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cc94f-8843-4845-818b-5fb601a50820 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:32.300616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cc94f-8843-4845-818b-5fb601a50820 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:32.301983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 664/1330] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:32.307762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2ea40-c44f-4d11-8518-545e66c312af None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:32.308191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2ea40-c44f-4d11-8518-545e66c312af None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:32.308305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2ea40-c44f-4d11-8518-545e66c312af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:32.375023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-76e2ea40-c44f-4d11-8518-545e66c312af None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:32.376222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-76e2ea40-c44f-4d11-8518-545e66c312af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:32.473683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 667/1331] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:32.767742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c65a46-26d2-4ad0-beba-aaf84db6e94b None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:32.768033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c65a46-26d2-4ad0-beba-aaf84db6e94b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:32.768310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c65a46-26d2-4ad0-beba-aaf84db6e94b None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:32.769727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 665/1332] 10.222.0.25 () {54 vars in 965 bytes} [Thu Sep 8 18:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:32.776271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f096-e438-4f83-b32f-521a9ceb6c0a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:32.776556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f096-e438-4f83-b32f-521a9ceb6c0a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:32.776826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f096-e438-4f83-b32f-521a9ceb6c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:32.822471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:32.853914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1ac2f096-e438-4f83-b32f-521a9ceb6c0a None None] MFA Rules not processed for user `bcf30ef695154d3a97d7e589aca21233`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:32.855811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1ac2f096-e438-4f83-b32f-521a9ceb6c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:32.938099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 668/1333] 10.222.0.25 () {58 vars in 1062 bytes} [Thu Sep 8 18:43:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:32.947998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c65a46-26d2-4ad0-beba-aaf84db6e94b None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:32.949573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-89c65a46-26d2-4ad0-beba-aaf84db6e94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:33.053635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c65a46-26d2-4ad0-beba-aaf84db6e94b None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:33.126959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6320c-7075-415a-81cf-be0d2e881b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:33.127494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6320c-7075-415a-81cf-be0d2e881b5d service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:33.127774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6320c-7075-415a-81cf-be0d2e881b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:33.128048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6320c-7075-415a-81cf-be0d2e881b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:33.129674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7ac6320c-7075-415a-81cf-be0d2e881b5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:33.158292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:33.239692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6320c-7075-415a-81cf-be0d2e881b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:33.241687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6320c-7075-415a-81cf-be0d2e881b5d service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:33.260018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 666/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:35.682594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6541f1d-d2f5-4d33-a022-5aa3fee2f755 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:35.683103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6541f1d-d2f5-4d33-a022-5aa3fee2f755 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:35.683197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6541f1d-d2f5-4d33-a022-5aa3fee2f755 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:35.684511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 669/1335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:35.689810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffbd43-6f21-4b8d-bb89-019bb992d527 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:35.690226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffbd43-6f21-4b8d-bb89-019bb992d527 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:35.690355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffbd43-6f21-4b8d-bb89-019bb992d527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:35.757083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2dffbd43-6f21-4b8d-bb89-019bb992d527 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:35.758275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2dffbd43-6f21-4b8d-bb89-019bb992d527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:35.832154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:35.854776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 667/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:36.143051 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6541f1d-d2f5-4d33-a022-5aa3fee2f755 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:36.144612 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e6541f1d-d2f5-4d33-a022-5aa3fee2f755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:36.247234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6541f1d-d2f5-4d33-a022-5aa3fee2f755 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:36.276341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:36.321677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1fa97-3873-4b20-99c1-cf6e6b30c24e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:36.322278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1fa97-3873-4b20-99c1-cf6e6b30c24e service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:36.322559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1fa97-3873-4b20-99c1-cf6e6b30c24e service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:36.322826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1fa97-3873-4b20-99c1-cf6e6b30c24e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:36.324395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5db1fa97-3873-4b20-99c1-cf6e6b30c24e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:36.432795 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1fa97-3873-4b20-99c1-cf6e6b30c24e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:36.434169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1fa97-3873-4b20-99c1-cf6e6b30c24e service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:36.451784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 670/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:39.292266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91530e11-f234-4f58-b197-7304710cc6be None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:39.292721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91530e11-f234-4f58-b197-7304710cc6be None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:39.292835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91530e11-f234-4f58-b197-7304710cc6be None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:39.294256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 668/1338] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:39.299261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b1a29-d4e3-41cf-a178-ec2459453ed9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:39.299696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b1a29-d4e3-41cf-a178-ec2459453ed9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:39.299795 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b1a29-d4e3-41cf-a178-ec2459453ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:39.366744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-5e1b1a29-d4e3-41cf-a178-ec2459453ed9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:39.367967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5e1b1a29-d4e3-41cf-a178-ec2459453ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:39.376129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:39.468818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 671/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:39.747069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91530e11-f234-4f58-b197-7304710cc6be None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:39.748614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-91530e11-f234-4f58-b197-7304710cc6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:39.855224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91530e11-f234-4f58-b197-7304710cc6be None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:39.929318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32732745-9022-48b2-a93c-90ec7233d3eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:39.929912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32732745-9022-48b2-a93c-90ec7233d3eb service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:39.930188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32732745-9022-48b2-a93c-90ec7233d3eb service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:39.930457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32732745-9022-48b2-a93c-90ec7233d3eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:39.932029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-32732745-9022-48b2-a93c-90ec7233d3eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:39.968699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:40.045519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32732745-9022-48b2-a93c-90ec7233d3eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:40.046949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32732745-9022-48b2-a93c-90ec7233d3eb service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:40.068695 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 669/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:44.560861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c39a9-df36-4238-92d7-4af4f43c9c07 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:44.561425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c39a9-df36-4238-92d7-4af4f43c9c07 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:44.561425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c39a9-df36-4238-92d7-4af4f43c9c07 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:44.562785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 672/1341] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:44.568111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e343b-34cd-4674-9bf2-26c776a56925 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:44.568551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e343b-34cd-4674-9bf2-26c776a56925 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:44.568928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e343b-34cd-4674-9bf2-26c776a56925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:44.639092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-4e7e343b-34cd-4674-9bf2-26c776a56925 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:44.640328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4e7e343b-34cd-4674-9bf2-26c776a56925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:44.729323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:44.740272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 670/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:45.030773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c39a9-df36-4238-92d7-4af4f43c9c07 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:45.032316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8b1c39a9-df36-4238-92d7-4af4f43c9c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:45.141937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c39a9-df36-4238-92d7-4af4f43c9c07 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:45.181601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:45.218519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb8b30-cf1e-49be-b80c-4b3a92fa5599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:45.219127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb8b30-cf1e-49be-b80c-4b3a92fa5599 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:45.219405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb8b30-cf1e-49be-b80c-4b3a92fa5599 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:45.219674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb8b30-cf1e-49be-b80c-4b3a92fa5599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:45.221318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-75eb8b30-cf1e-49be-b80c-4b3a92fa5599 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:45.341762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb8b30-cf1e-49be-b80c-4b3a92fa5599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:45.343202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb8b30-cf1e-49be-b80c-4b3a92fa5599 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:45.362757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 673/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:48.528198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cef502-411b-4ca6-8419-8e72abb5c428 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:48.528772 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cef502-411b-4ca6-8419-8e72abb5c428 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:48.528772 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cef502-411b-4ca6-8419-8e72abb5c428 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:48.530170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 671/1344] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:48.535493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9bcf8-6790-47a0-9bfa-3f3d16e04ff5 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:48.535897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9bcf8-6790-47a0-9bfa-3f3d16e04ff5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:48.536038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9bcf8-6790-47a0-9bfa-3f3d16e04ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:48.626320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-30f9bcf8-6790-47a0-9bfa-3f3d16e04ff5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:48.627542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-30f9bcf8-6790-47a0-9bfa-3f3d16e04ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:48.646976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:48.735820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 674/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:49.022451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cef502-411b-4ca6-8419-8e72abb5c428 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:49.024178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-65cef502-411b-4ca6-8419-8e72abb5c428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:49.133544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cef502-411b-4ca6-8419-8e72abb5c428 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:49.211334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d28258-5233-483e-8f56-4621b21381e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:49.212027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28258-5233-483e-8f56-4621b21381e5 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:49.212242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28258-5233-483e-8f56-4621b21381e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:49.212513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28258-5233-483e-8f56-4621b21381e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:49.214132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-59d28258-5233-483e-8f56-4621b21381e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:49.259978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:49.358842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d28258-5233-483e-8f56-4621b21381e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:49.360292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d28258-5233-483e-8f56-4621b21381e5 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:49.381833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 672/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:53.760333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b9c6-ecfc-4f4a-9cda-3cd30fa7a5dc None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:53.760871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b9c6-ecfc-4f4a-9cda-3cd30fa7a5dc None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:53.760871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750b9c6-ecfc-4f4a-9cda-3cd30fa7a5dc None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:53.762286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 675/1347] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:53.767697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5adc1-a1a5-4a8e-99ff-adaff3c31df5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:53.768168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5adc1-a1a5-4a8e-99ff-adaff3c31df5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:53.768277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5adc1-a1a5-4a8e-99ff-adaff3c31df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:53.840179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-1ce5adc1-a1a5-4a8e-99ff-adaff3c31df5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:53.841562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1ce5adc1-a1a5-4a8e-99ff-adaff3c31df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:53.943916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:53.946224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 673/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:43:54.227474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b9c6-ecfc-4f4a-9cda-3cd30fa7a5dc None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:43:54.229329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c750b9c6-ecfc-4f4a-9cda-3cd30fa7a5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:54.334382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750b9c6-ecfc-4f4a-9cda-3cd30fa7a5dc None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:43:54.383018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:54.408458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39507a-1f08-4cad-bdf8-e19499593617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:43:54.409089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39507a-1f08-4cad-bdf8-e19499593617 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:54.409390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39507a-1f08-4cad-bdf8-e19499593617 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:54.409648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39507a-1f08-4cad-bdf8-e19499593617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:54.411215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ce39507a-1f08-4cad-bdf8-e19499593617 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:54.519983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39507a-1f08-4cad-bdf8-e19499593617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:43:54.521962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39507a-1f08-4cad-bdf8-e19499593617 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:43:54.540643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 676/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:43:59.550504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c317e-70d3-44c7-a310-f9f4d0e76964 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:59.550974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c317e-70d3-44c7-a310-f9f4d0e76964 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:59.551073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c317e-70d3-44c7-a310-f9f4d0e76964 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:59.553251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 674/1350] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:43:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:43:59.558948 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab355f9-43b1-4dfe-9422-59343e0f5975 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:43:59.558948 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab355f9-43b1-4dfe-9422-59343e0f5975 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:43:59.559402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab355f9-43b1-4dfe-9422-59343e0f5975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:43:59.689958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2ab355f9-43b1-4dfe-9422-59343e0f5975 None None] MFA Rules not processed for user `a074714e365648b88c9d3a0495a17b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:43:59.691174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2ab355f9-43b1-4dfe-9422-59343e0f5975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:43:59.712059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:43:59.865777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 677/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:43:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:04.237638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22008893-a680-470c-a0fa-f0380e9b49e7 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:04.238165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22008893-a680-470c-a0fa-f0380e9b49e7 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:04.238165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22008893-a680-470c-a0fa-f0380e9b49e7 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:04.240411 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 675/1352] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:44:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:04.245473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7cad6-c9f1-4739-abb7-7d417f52909c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:04.245953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7cad6-c9f1-4739-abb7-7d417f52909c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:04.246040 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7cad6-c9f1-4739-abb7-7d417f52909c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:04.328347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-56b7cad6-c9f1-4739-abb7-7d417f52909c None None] MFA Rules not processed for user `a074714e365648b88c9d3a0495a17b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:04.329608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-56b7cad6-c9f1-4739-abb7-7d417f52909c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:04.410876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:04.452104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 678/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:44:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:05.737961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08057a5-bfd5-43f6-a9a9-111d16f0e219 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:05.738478 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08057a5-bfd5-43f6-a9a9-111d16f0e219 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:05.738567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08057a5-bfd5-43f6-a9a9-111d16f0e219 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:05.739901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 676/1354] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:05.745195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f164459-6ea4-4645-b36d-5809d88e9f02 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:05.745642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f164459-6ea4-4645-b36d-5809d88e9f02 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:05.745759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f164459-6ea4-4645-b36d-5809d88e9f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:05.824382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-0f164459-6ea4-4645-b36d-5809d88e9f02 None None] MFA Rules not processed for user `a074714e365648b88c9d3a0495a17b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:05.825620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0f164459-6ea4-4645-b36d-5809d88e9f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:05.945479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 679/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:44:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:09.722041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec4aee-4f9a-475a-b462-6216981c7885 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:09.723313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec4aee-4f9a-475a-b462-6216981c7885 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:09.724479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec4aee-4f9a-475a-b462-6216981c7885 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:09.727091 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 677/1356] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:09.732553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be5f48-58b3-47ec-87d0-eedd1df388ab None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:09.733722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be5f48-58b3-47ec-87d0-eedd1df388ab None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:09.734227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be5f48-58b3-47ec-87d0-eedd1df388ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:09.790256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:09.824311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-57be5f48-58b3-47ec-87d0-eedd1df388ab None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:09.825537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-57be5f48-58b3-47ec-87d0-eedd1df388ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:09.937117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 680/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:10.239377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a3f6d-1854-4e29-ae90-b5b9b65c7958 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:10.240029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a3f6d-1854-4e29-ae90-b5b9b65c7958 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:10.240651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a3f6d-1854-4e29-ae90-b5b9b65c7958 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:10.242617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 678/1358] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 8 18:44:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:10.252977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c36dd-8d84-4852-bfd0-855f0d18ff27 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:10.253851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c36dd-8d84-4852-bfd0-855f0d18ff27 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:10.254562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c36dd-8d84-4852-bfd0-855f0d18ff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:10.338593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-db7c36dd-8d84-4852-bfd0-855f0d18ff27 None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:10.339903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-db7c36dd-8d84-4852-bfd0-855f0d18ff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:10.394042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:10.468310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 681/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 8 18:44:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:10.477746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a3f6d-1854-4e29-ae90-b5b9b65c7958 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:10.479764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-878a3f6d-1854-4e29-ae90-b5b9b65c7958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:10.640328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a3f6d-1854-4e29-ae90-b5b9b65c7958 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:10.723195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:10.767168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958548e-dfa8-48d8-ade1-266ccc63bc17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:10.767947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958548e-dfa8-48d8-ade1-266ccc63bc17 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:10.768334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958548e-dfa8-48d8-ade1-266ccc63bc17 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:10.768711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958548e-dfa8-48d8-ade1-266ccc63bc17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:10.770550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3958548e-dfa8-48d8-ade1-266ccc63bc17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:10.880078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958548e-dfa8-48d8-ade1-266ccc63bc17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:10.882564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958548e-dfa8-48d8-ade1-266ccc63bc17 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:10.902610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 679/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:44:13.323208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ef628-08b8-47a6-aaed-2ebc193f80b4 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:13.323668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ef628-08b8-47a6-aaed-2ebc193f80b4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:13.324064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ef628-08b8-47a6-aaed-2ebc193f80b4 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:13.326056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 682/1361] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:44:13.331266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e418af-11d9-4d34-948c-98475984e5f5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:13.331658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e418af-11d9-4d34-948c-98475984e5f5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:13.331992 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e418af-11d9-4d34-948c-98475984e5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:13.402806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-21e418af-11d9-4d34-948c-98475984e5f5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:13.404155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-21e418af-11d9-4d34-948c-98475984e5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:13.444574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:13.510792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 680/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:13.809558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ef628-08b8-47a6-aaed-2ebc193f80b4 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:13.811192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-624ef628-08b8-47a6-aaed-2ebc193f80b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:14.000318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ef628-08b8-47a6-aaed-2ebc193f80b4 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:14.013106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:14.145448 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15f522-6a7c-432b-8ab7-19d82599dcf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:14.145804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15f522-6a7c-432b-8ab7-19d82599dcf2 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:14.146090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15f522-6a7c-432b-8ab7-19d82599dcf2 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:14.146359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15f522-6a7c-432b-8ab7-19d82599dcf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:14.148075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5a15f522-6a7c-432b-8ab7-19d82599dcf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:14.244965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15f522-6a7c-432b-8ab7-19d82599dcf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:14.246706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15f522-6a7c-432b-8ab7-19d82599dcf2 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:14.266501 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 683/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:16.920054 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba49de1-df43-4f44-877d-7c2355eefbb5 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:16.920573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba49de1-df43-4f44-877d-7c2355eefbb5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:16.920926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba49de1-df43-4f44-877d-7c2355eefbb5 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:16.923721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 681/1364] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:16.932033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814a65a-e186-47a8-8050-c58e2821773f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:16.932509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814a65a-e186-47a8-8050-c58e2821773f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:16.932591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814a65a-e186-47a8-8050-c58e2821773f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:16.949169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:17.003275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d814a65a-e186-47a8-8050-c58e2821773f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:17.004478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d814a65a-e186-47a8-8050-c58e2821773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:17.109010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 684/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:17.398562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eda4aa-eed6-47b1-922a-1e7aa06f0827 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:17.398924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eda4aa-eed6-47b1-922a-1e7aa06f0827 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:17.399192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eda4aa-eed6-47b1-922a-1e7aa06f0827 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:17.400638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 682/1366] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 8 18:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:17.405994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e74b3-4a5e-44c5-be1d-bd2ea36f5f6b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:17.412235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e74b3-4a5e-44c5-be1d-bd2ea36f5f6b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:17.412235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e74b3-4a5e-44c5-be1d-bd2ea36f5f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:17.492412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d32e74b3-4a5e-44c5-be1d-bd2ea36f5f6b None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:17.494584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d32e74b3-4a5e-44c5-be1d-bd2ea36f5f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:17.535738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:17.638471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 685/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 8 18:44:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:17.647335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eda4aa-eed6-47b1-922a-1e7aa06f0827 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:17.648923 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-11eda4aa-eed6-47b1-922a-1e7aa06f0827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:17.818132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eda4aa-eed6-47b1-922a-1e7aa06f0827 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:17.843775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:18.052389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d92ec7-a823-4b2c-a69d-cd8de4cb618c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:18.052389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d92ec7-a823-4b2c-a69d-cd8de4cb618c service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:18.052389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d92ec7-a823-4b2c-a69d-cd8de4cb618c service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:18.052389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d92ec7-a823-4b2c-a69d-cd8de4cb618c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:18.053444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-75d92ec7-a823-4b2c-a69d-cd8de4cb618c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:18.053541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6e795-4a87-412f-8e41-e7c6e3768e97 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:18.053541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6e795-4a87-412f-8e41-e7c6e3768e97 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:18.053541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6e795-4a87-412f-8e41-e7c6e3768e97 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:18.053541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 686/1368] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:44:18.053541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82bb53-4586-43d4-9f2f-df4ee5d51b6c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:18.053541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82bb53-4586-43d4-9f2f-df4ee5d51b6c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:18.054329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82bb53-4586-43d4-9f2f-df4ee5d51b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:18.073192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d92ec7-a823-4b2c-a69d-cd8de4cb618c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:18.075325 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d92ec7-a823-4b2c-a69d-cd8de4cb618c service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:18.080545 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2e82bb53-4586-43d4-9f2f-df4ee5d51b6c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:18.082265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2e82bb53-4586-43d4-9f2f-df4ee5d51b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:18.100414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 683/1369] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:18.189013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:18.197630 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 687/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:44:18.547793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a3bae-84be-4a0c-8718-edd52a59f14b None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:18.548084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a3bae-84be-4a0c-8718-edd52a59f14b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:18.548502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a3bae-84be-4a0c-8718-edd52a59f14b None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:18.550130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 684/1371] 10.222.0.25 () {54 vars in 964 bytes} [Thu Sep 8 18:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:18.559692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bd44d-4dd6-4317-bac5-1aa60056dac7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:18.559692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bd44d-4dd6-4317-bac5-1aa60056dac7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:18.559692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bd44d-4dd6-4317-bac5-1aa60056dac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:18.657112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-3d8bd44d-4dd6-4317-bac5-1aa60056dac7 None None] MFA Rules not processed for user `605b5d4c0fc9409a94dad22c77c67332`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:18.658114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3d8bd44d-4dd6-4317-bac5-1aa60056dac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:18.742726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 688/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Sep 8 18:44:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:18.750266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a3bae-84be-4a0c-8718-edd52a59f14b None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:18.751800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-611a3bae-84be-4a0c-8718-edd52a59f14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:18.782882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:18.862164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a3bae-84be-4a0c-8718-edd52a59f14b None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:18.956528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe4702-a180-402e-a44f-0bcc9b348b5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:18.957140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe4702-a180-402e-a44f-0bcc9b348b5c service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:18.957420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe4702-a180-402e-a44f-0bcc9b348b5c service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:18.957689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe4702-a180-402e-a44f-0bcc9b348b5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:18.959329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7bbe4702-a180-402e-a44f-0bcc9b348b5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:19.044761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:19.076196 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe4702-a180-402e-a44f-0bcc9b348b5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:19.077665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe4702-a180-402e-a44f-0bcc9b348b5c service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:19.101000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 685/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:44:20.561695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d749ee-997c-450f-a0c3-1f18e350cba3 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:20.562156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d749ee-997c-450f-a0c3-1f18e350cba3 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:20.562249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d749ee-997c-450f-a0c3-1f18e350cba3 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:20.563611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 689/1374] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:20.569631 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bca5fb-16ed-4bf9-a5c5-14021c62ca8b None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:20.570090 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bca5fb-16ed-4bf9-a5c5-14021c62ca8b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:20.570200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bca5fb-16ed-4bf9-a5c5-14021c62ca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:20.651181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-03bca5fb-16ed-4bf9-a5c5-14021c62ca8b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:20.652440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-03bca5fb-16ed-4bf9-a5c5-14021c62ca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:20.756983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 686/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:21.056392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d749ee-997c-450f-a0c3-1f18e350cba3 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:21.057934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c4d749ee-997c-450f-a0c3-1f18e350cba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:21.149784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:21.219101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d749ee-997c-450f-a0c3-1f18e350cba3 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:21.346345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3a1ba-dedd-455c-8fca-f82bd73945b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:21.346888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3a1ba-dedd-455c-8fca-f82bd73945b9 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:21.347169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3a1ba-dedd-455c-8fca-f82bd73945b9 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:21.347444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3a1ba-dedd-455c-8fca-f82bd73945b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:21.349677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1ac3a1ba-dedd-455c-8fca-f82bd73945b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:21.403143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:21.448904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3a1ba-dedd-455c-8fca-f82bd73945b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:21.450777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3a1ba-dedd-455c-8fca-f82bd73945b9 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:21.470249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 690/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:23.872534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef3f4f-5d55-4eba-ad63-37c7899871bb None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:23.873086 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef3f4f-5d55-4eba-ad63-37c7899871bb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:23.873086 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef3f4f-5d55-4eba-ad63-37c7899871bb None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:23.874518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 687/1377] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:23.880263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ace62-0b5e-4576-be50-74efb15c58b7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:23.880695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ace62-0b5e-4576-be50-74efb15c58b7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:23.880829 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ace62-0b5e-4576-be50-74efb15c58b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:23.953274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-330ace62-0b5e-4576-be50-74efb15c58b7 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:23.954468 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-330ace62-0b5e-4576-be50-74efb15c58b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:24.059159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 691/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:24.349871 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef3f4f-5d55-4eba-ad63-37c7899871bb None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:24.351395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-acef3f4f-5d55-4eba-ad63-37c7899871bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:24.390315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:24.507529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef3f4f-5d55-4eba-ad63-37c7899871bb None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:24.633839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:24.639210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526571a9-54ba-406e-a8aa-8c2464bbcee8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:24.639825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526571a9-54ba-406e-a8aa-8c2464bbcee8 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:24.640102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526571a9-54ba-406e-a8aa-8c2464bbcee8 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:24.640509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526571a9-54ba-406e-a8aa-8c2464bbcee8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:24.642267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-526571a9-54ba-406e-a8aa-8c2464bbcee8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:24.742587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526571a9-54ba-406e-a8aa-8c2464bbcee8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:24.744308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526571a9-54ba-406e-a8aa-8c2464bbcee8 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:24.762571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 688/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:27.195163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecda74-73f8-4485-86cf-f9daf5702c4d None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:27.195823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecda74-73f8-4485-86cf-f9daf5702c4d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:27.195823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecda74-73f8-4485-86cf-f9daf5702c4d None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:27.197513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 692/1380] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:27.204066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edbd23-184c-48bf-b0bc-d0528436ff68 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:27.204860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edbd23-184c-48bf-b0bc-d0528436ff68 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:27.205193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edbd23-184c-48bf-b0bc-d0528436ff68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:27.277645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c1edbd23-184c-48bf-b0bc-d0528436ff68 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:27.278929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c1edbd23-184c-48bf-b0bc-d0528436ff68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:27.359380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:27.392896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 689/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:27.694761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecda74-73f8-4485-86cf-f9daf5702c4d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:27.696794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-22ecda74-73f8-4485-86cf-f9daf5702c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:27.732509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:27.981783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecda74-73f8-4485-86cf-f9daf5702c4d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:28.100640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:28.118742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1c055-7313-4bd7-86b0-f53ad7ae6d06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:28.119390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1c055-7313-4bd7-86b0-f53ad7ae6d06 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:28.119686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1c055-7313-4bd7-86b0-f53ad7ae6d06 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:28.119991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1c055-7313-4bd7-86b0-f53ad7ae6d06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:28.121708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-57d1c055-7313-4bd7-86b0-f53ad7ae6d06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:28.234010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1c055-7313-4bd7-86b0-f53ad7ae6d06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:28.235983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1c055-7313-4bd7-86b0-f53ad7ae6d06 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:28.260364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 693/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:30.765697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7df3f-6605-4e28-bb42-b3f1815860d5 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:30.766384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7df3f-6605-4e28-bb42-b3f1815860d5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:30.766384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7df3f-6605-4e28-bb42-b3f1815860d5 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:30.768134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 690/1383] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:30.774280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b630d-0fc3-4b4e-82f8-4e87bf2ba34a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:30.774749 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b630d-0fc3-4b4e-82f8-4e87bf2ba34a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:30.774971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b630d-0fc3-4b4e-82f8-4e87bf2ba34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:30.851540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-513b630d-0fc3-4b4e-82f8-4e87bf2ba34a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:30.852785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-513b630d-0fc3-4b4e-82f8-4e87bf2ba34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:30.918894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:30.963333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 694/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:31.257158 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7df3f-6605-4e28-bb42-b3f1815860d5 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:31.259154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6fd7df3f-6605-4e28-bb42-b3f1815860d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:31.423782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7df3f-6605-4e28-bb42-b3f1815860d5 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:31.487501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:31.569815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c65bd-bef0-43c5-a3f5-e76a3bdf5aef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:31.570650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c65bd-bef0-43c5-a3f5-e76a3bdf5aef service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:31.571020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c65bd-bef0-43c5-a3f5-e76a3bdf5aef service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:31.571402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c65bd-bef0-43c5-a3f5-e76a3bdf5aef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:31.573773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-083c65bd-bef0-43c5-a3f5-e76a3bdf5aef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:31.677331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c65bd-bef0-43c5-a3f5-e76a3bdf5aef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:31.679181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c65bd-bef0-43c5-a3f5-e76a3bdf5aef service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:31.700222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 691/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:33.997434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a80b95-bd41-4b92-80c0-aad51895def7 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:33.997954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a80b95-bd41-4b92-80c0-aad51895def7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:33.997954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a80b95-bd41-4b92-80c0-aad51895def7 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:33.999299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 695/1386] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:34.005334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06122cad-0113-4039-a7ca-8d97ffd25144 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:34.005889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06122cad-0113-4039-a7ca-8d97ffd25144 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:34.006029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06122cad-0113-4039-a7ca-8d97ffd25144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:34.073665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-06122cad-0113-4039-a7ca-8d97ffd25144 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:34.074874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-06122cad-0113-4039-a7ca-8d97ffd25144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:34.083192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:34.171443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 692/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:34.464466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a80b95-bd41-4b92-80c0-aad51895def7 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:34.466413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f1a80b95-bd41-4b92-80c0-aad51895def7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:34.624286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a80b95-bd41-4b92-80c0-aad51895def7 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:34.681464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:34.756381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef393a-1263-4e42-b2d9-af20bf7fd411 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:34.756919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef393a-1263-4e42-b2d9-af20bf7fd411 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:34.757231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef393a-1263-4e42-b2d9-af20bf7fd411 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:34.757506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef393a-1263-4e42-b2d9-af20bf7fd411 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:34.759209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2bef393a-1263-4e42-b2d9-af20bf7fd411 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:34.856146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef393a-1263-4e42-b2d9-af20bf7fd411 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:34.857787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef393a-1263-4e42-b2d9-af20bf7fd411 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:34.868700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb682a57-a9a7-4ee0-9f27-a3f2cf8c3416 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:34.868989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb682a57-a9a7-4ee0-9f27-a3f2cf8c3416 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:34.869292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb682a57-a9a7-4ee0-9f27-a3f2cf8c3416 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:34.870651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 693/1388] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:44:34.875467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8813c-1a73-4098-80a1-fcc111763740 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:34.875748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8813c-1a73-4098-80a1-fcc111763740 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:34.876050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8813c-1a73-4098-80a1-fcc111763740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:34.876637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 696/1389] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:34.943807 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-75c8813c-1a73-4098-80a1-fcc111763740 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:34.945093 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-75c8813c-1a73-4098-80a1-fcc111763740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:35.034254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:35.051259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 694/1390] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:44:35.358704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837d217-619f-4668-85a5-145fdd41164f None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:35.359199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837d217-619f-4668-85a5-145fdd41164f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:35.359293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837d217-619f-4668-85a5-145fdd41164f None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:35.360637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 697/1391] 10.222.0.25 () {54 vars in 966 bytes} [Thu Sep 8 18:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:44:35.366540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c9f05-07d5-4b6c-9a4f-c20b30d6f3f9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:35.366833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c9f05-07d5-4b6c-9a4f-c20b30d6f3f9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:35.367111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c9f05-07d5-4b6c-9a4f-c20b30d6f3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:35.447669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-687c9f05-07d5-4b6c-9a4f-c20b30d6f3f9 None None] MFA Rules not processed for user `c8d10585480f41608bc6935e282f7e27`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:35.449842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-687c9f05-07d5-4b6c-9a4f-c20b30d6f3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:35.531054 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 695/1392] 10.222.0.25 () {58 vars in 1063 bytes} [Thu Sep 8 18:44:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:44:35.540889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837d217-619f-4668-85a5-145fdd41164f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:35.542448 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1837d217-619f-4668-85a5-145fdd41164f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:35.600041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:35.653293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837d217-619f-4668-85a5-145fdd41164f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:35.728057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7fc7d-6506-4523-abdb-b33edeb749f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d10585480f41608bc6935e282f7e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:35.728931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7fc7d-6506-4523-abdb-b33edeb749f8 service glance] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:35.729250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7fc7d-6506-4523-abdb-b33edeb749f8 service glance] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:35.729523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7fc7d-6506-4523-abdb-b33edeb749f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:35.731129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b4b7fc7d-6506-4523-abdb-b33edeb749f8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:35.830301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:35.843032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7fc7d-6506-4523-abdb-b33edeb749f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:35.844453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7fc7d-6506-4523-abdb-b33edeb749f8 service glance] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:35.863627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 698/1393] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:36.794125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058cb2c-2811-4c87-bf23-fb494186a6f4 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:36.794599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058cb2c-2811-4c87-bf23-fb494186a6f4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:36.794888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058cb2c-2811-4c87-bf23-fb494186a6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:36.985551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-0058cb2c-2811-4c87-bf23-fb494186a6f4 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:36.986705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0058cb2c-2811-4c87-bf23-fb494186a6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:37.023973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:37.080214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 696/1394] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:44:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:44:37.131216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471d868-c508-4bdc-98f5-914523045991 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:37.131746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471d868-c508-4bdc-98f5-914523045991 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:37.131839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471d868-c508-4bdc-98f5-914523045991 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:37.133164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 699/1395] 10.222.0.25 () {54 vars in 943 bytes} [Thu Sep 8 18:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:37.139462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fe451-d422-4442-9ca3-82ad06bbee6c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:37.139828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fe451-d422-4442-9ca3-82ad06bbee6c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:37.140169 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fe451-d422-4442-9ca3-82ad06bbee6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:37.218328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09437a-f292-48af-beab-a5acca05d9f5 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:37.218630 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09437a-f292-48af-beab-a5acca05d9f5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:37.218874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09437a-f292-48af-beab-a5acca05d9f5 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:37.219058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-861fe451-d422-4442-9ca3-82ad06bbee6c None None] MFA Rules not processed for user `88a9b2cee21f4831b221d113ddbbd29d`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:37.220201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 700/1396] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:37.220380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-861fe451-d422-4442-9ca3-82ad06bbee6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:37.225141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6245910-edde-430f-946b-7da50887011f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:37.225428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6245910-edde-430f-946b-7da50887011f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:37.225701 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6245910-edde-430f-946b-7da50887011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:37.292757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e6245910-edde-430f-946b-7da50887011f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:37.294017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e6245910-edde-430f-946b-7da50887011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:37.332019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 697/1397] 10.222.0.25 () {58 vars in 1040 bytes} [Thu Sep 8 18:44:37 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:37.341712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fe451-d422-4442-9ca3-82ad06bbee6c None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:37.352323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fe451-d422-4442-9ca3-82ad06bbee6c None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:37.355556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:37.394865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 701/1398] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:44:37.457261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a647a7b-e221-4f3e-9f15-d82b8803d5f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:37.457906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a647a7b-e221-4f3e-9f15-d82b8803d5f9 service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:37.458177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a647a7b-e221-4f3e-9f15-d82b8803d5f9 service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:37.458450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a647a7b-e221-4f3e-9f15-d82b8803d5f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:37.460061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0a647a7b-e221-4f3e-9f15-d82b8803d5f9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:37.544505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a647a7b-e221-4f3e-9f15-d82b8803d5f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:37.546092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a647a7b-e221-4f3e-9f15-d82b8803d5f9 service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:37.554769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 698/1399] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:44:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 18:44:37.687386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6245910-edde-430f-946b-7da50887011f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:37.688594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e6245910-edde-430f-946b-7da50887011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:37.752547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:37.804851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6245910-edde-430f-946b-7da50887011f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:37.924024 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69479db-1c68-4b60-a7d5-3e8b1ec0886a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:37.924633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69479db-1c68-4b60-a7d5-3e8b1ec0886a service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:37.924913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69479db-1c68-4b60-a7d5-3e8b1ec0886a service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:37.925229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69479db-1c68-4b60-a7d5-3e8b1ec0886a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:37.927156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b69479db-1c68-4b60-a7d5-3e8b1ec0886a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:37.987734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:38.042131 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69479db-1c68-4b60-a7d5-3e8b1ec0886a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:38.043830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69479db-1c68-4b60-a7d5-3e8b1ec0886a service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:38.063117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 702/1400] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:40.434895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607120f-dde0-462b-ad9e-d3b9551ffbb1 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:40.435529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607120f-dde0-462b-ad9e-d3b9551ffbb1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:40.435529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607120f-dde0-462b-ad9e-d3b9551ffbb1 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:40.436803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 699/1401] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:40.443054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a0f3-887d-49bd-b9e6-fd5e24f233b8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:40.443575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a0f3-887d-49bd-b9e6-fd5e24f233b8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:40.443575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4a0f3-887d-49bd-b9e6-fd5e24f233b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:40.514960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a1b4a0f3-887d-49bd-b9e6-fd5e24f233b8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:40.516344 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a1b4a0f3-887d-49bd-b9e6-fd5e24f233b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:40.637161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 703/1402] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:40.923483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607120f-dde0-462b-ad9e-d3b9551ffbb1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:40.927084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3607120f-dde0-462b-ad9e-d3b9551ffbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:40.971443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:41.095274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607120f-dde0-462b-ad9e-d3b9551ffbb1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:41.218499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:41.224632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e816c-1662-416c-82b4-f350551c5767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:41.225278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e816c-1662-416c-82b4-f350551c5767 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:41.225566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e816c-1662-416c-82b4-f350551c5767 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:41.225843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e816c-1662-416c-82b4-f350551c5767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:41.227694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-433e816c-1662-416c-82b4-f350551c5767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:41.336968 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e816c-1662-416c-82b4-f350551c5767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:41.338631 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e816c-1662-416c-82b4-f350551c5767 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:41.359278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 700/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:43.715715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dca14a-81e9-46ae-9267-0b24c0b45905 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:43.716272 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dca14a-81e9-46ae-9267-0b24c0b45905 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:43.716272 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dca14a-81e9-46ae-9267-0b24c0b45905 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:43.717813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 704/1404] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:43.724292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82861c5a-39d9-4537-9a0f-9f244c90970e None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:43.724811 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82861c5a-39d9-4537-9a0f-9f244c90970e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:43.725228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82861c5a-39d9-4537-9a0f-9f244c90970e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:43.797577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-82861c5a-39d9-4537-9a0f-9f244c90970e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:43.799414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-82861c5a-39d9-4537-9a0f-9f244c90970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:43.872143 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:43.907603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 701/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:44.206873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dca14a-81e9-46ae-9267-0b24c0b45905 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:44.209608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c1dca14a-81e9-46ae-9267-0b24c0b45905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:44.279136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:44.415562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dca14a-81e9-46ae-9267-0b24c0b45905 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:44.569584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf5d0e-f970-4ce8-936a-ad65b012149f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:44.570387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf5d0e-f970-4ce8-936a-ad65b012149f service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:44.570942 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf5d0e-f970-4ce8-936a-ad65b012149f service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:44.571403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf5d0e-f970-4ce8-936a-ad65b012149f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:44.573269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6acf5d0e-f970-4ce8-936a-ad65b012149f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:44.691378 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf5d0e-f970-4ce8-936a-ad65b012149f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:44.693575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf5d0e-f970-4ce8-936a-ad65b012149f service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:44.712510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 705/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:44.912295 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:47.303156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafc1df-dcdd-4972-bfda-00131d60b424 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:47.303721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafc1df-dcdd-4972-bfda-00131d60b424 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:47.303721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafc1df-dcdd-4972-bfda-00131d60b424 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:47.305550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 702/1407] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:44:47.311000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5902f-2917-4612-b827-0c90943a8a88 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:47.311419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5902f-2917-4612-b827-0c90943a8a88 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:47.313028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5902f-2917-4612-b827-0c90943a8a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:47.387518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f2d5902f-2917-4612-b827-0c90943a8a88 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:44:47.388778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f2d5902f-2917-4612-b827-0c90943a8a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:47.458651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:47.496487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 706/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:44:47.787576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafc1df-dcdd-4972-bfda-00131d60b424 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:44:47.789100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cdafc1df-dcdd-4972-bfda-00131d60b424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:47.951815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafc1df-dcdd-4972-bfda-00131d60b424 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:44:48.010848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:48.081375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06feffd8-05c6-4891-8032-c6f993c0eb58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:44:48.081989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feffd8-05c6-4891-8032-c6f993c0eb58 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:44:48.082280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feffd8-05c6-4891-8032-c6f993c0eb58 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:44:48.082618 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06feffd8-05c6-4891-8032-c6f993c0eb58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:44:48.084590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-06feffd8-05c6-4891-8032-c6f993c0eb58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:44:48.187532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06feffd8-05c6-4891-8032-c6f993c0eb58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:44:48.189169 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06feffd8-05c6-4891-8032-c6f993c0eb58 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:44:48.208556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 703/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:44:57.495113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=13 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:44:58.205832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=74 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:01.441649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9fff7-b809-4926-8085-d6f4e8635670 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:01.443266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9fff7-b809-4926-8085-d6f4e8635670 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:01.443266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9fff7-b809-4926-8085-d6f4e8635670 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:01.444088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 707/1410] 10.222.0.25 () {54 vars in 918 bytes} [Thu Sep 8 18:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:01.449328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931443b9-4944-4237-adee-051b81aeaa1b None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:01.449795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931443b9-4944-4237-adee-051b81aeaa1b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:01.457171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931443b9-4944-4237-adee-051b81aeaa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:01.530200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-931443b9-4944-4237-adee-051b81aeaa1b None None] MFA Rules not processed for user `a074714e365648b88c9d3a0495a17b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:01.531421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-931443b9-4944-4237-adee-051b81aeaa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:01.641409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 704/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 8 18:45:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:01.656163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7493a6e-9aa5-46be-ad84-dec0c7d28b2e None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:01.656449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7493a6e-9aa5-46be-ad84-dec0c7d28b2e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:01.656727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7493a6e-9aa5-46be-ad84-dec0c7d28b2e None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:01.658138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 708/1412] 10.222.0.25 () {54 vars in 918 bytes} [Thu Sep 8 18:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:01.662770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2ef16-04e0-4b19-8606-20d64ff2212d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:01.663084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2ef16-04e0-4b19-8606-20d64ff2212d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:01.663356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2ef16-04e0-4b19-8606-20d64ff2212d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:01.735825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:01.738397 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-5be2ef16-04e0-4b19-8606-20d64ff2212d None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:01.740140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5be2ef16-04e0-4b19-8606-20d64ff2212d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:01.858048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 705/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 8 18:45:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:04.459465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d38423-208a-4e05-95b6-a70422b643fc None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:04.459947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d38423-208a-4e05-95b6-a70422b643fc None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:04.460044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d38423-208a-4e05-95b6-a70422b643fc None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:04.461423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:04.466928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd4f80-6881-4b66-9d43-48135b1677e4 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:04.471333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd4f80-6881-4b66-9d43-48135b1677e4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:04.471333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd4f80-6881-4b66-9d43-48135b1677e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:04.534651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f8dd4f80-6881-4b66-9d43-48135b1677e4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:04.535926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f8dd4f80-6881-4b66-9d43-48135b1677e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:04.592200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:04.637810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:04.938230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d38423-208a-4e05-95b6-a70422b643fc None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:04.939766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-17d38423-208a-4e05-95b6-a70422b643fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:05.091150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d38423-208a-4e05-95b6-a70422b643fc None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:05.165301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:05.205972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc39a0f-854e-43d0-be4c-d0e7ea72e86e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:05.206513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc39a0f-854e-43d0-be4c-d0e7ea72e86e service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:05.206797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc39a0f-854e-43d0-be4c-d0e7ea72e86e service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:05.207070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc39a0f-854e-43d0-be4c-d0e7ea72e86e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:05.209312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-bbc39a0f-854e-43d0-be4c-d0e7ea72e86e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:05.298854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc39a0f-854e-43d0-be4c-d0e7ea72e86e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:05.300441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc39a0f-854e-43d0-be4c-d0e7ea72e86e service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:05.323657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 710/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:45:06.479559 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4141393-7c3b-46d4-9640-4fabd3d9fbe4 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:06.480137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4141393-7c3b-46d4-9640-4fabd3d9fbe4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:06.480137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4141393-7c3b-46d4-9640-4fabd3d9fbe4 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:06.481430 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:06.487542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e686c0-f308-4a41-910a-8d03273c57bf None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:06.487959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e686c0-f308-4a41-910a-8d03273c57bf None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:06.488087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e686c0-f308-4a41-910a-8d03273c57bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:06.565640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-97e686c0-f308-4a41-910a-8d03273c57bf None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:06.566942 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-97e686c0-f308-4a41-910a-8d03273c57bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:06.594923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:06.667831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:45:06.830207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4141393-7c3b-46d4-9640-4fabd3d9fbe4 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:06.831714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f4141393-7c3b-46d4-9640-4fabd3d9fbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:06.979764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4141393-7c3b-46d4-9640-4fabd3d9fbe4 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:07.019505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:07.091724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d58fdb-7cec-4894-b98a-79ad11fd3811 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:07.092554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d58fdb-7cec-4894-b98a-79ad11fd3811 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:07.093095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d58fdb-7cec-4894-b98a-79ad11fd3811 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:07.093836 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d58fdb-7cec-4894-b98a-79ad11fd3811 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:07.095542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-47d58fdb-7cec-4894-b98a-79ad11fd3811 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:07.191421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d58fdb-7cec-4894-b98a-79ad11fd3811 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:07.193646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d58fdb-7cec-4894-b98a-79ad11fd3811 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:07.212046 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 708/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:08.351397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097efc6-d6b3-409f-9f6d-8d8f223b1e6c None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:08.351968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097efc6-d6b3-409f-9f6d-8d8f223b1e6c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:08.351968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097efc6-d6b3-409f-9f6d-8d8f223b1e6c None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:08.353339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:08.359663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598f0e1-28ed-4abb-8af2-66d0f5e0f494 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:08.360146 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598f0e1-28ed-4abb-8af2-66d0f5e0f494 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:08.360252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598f0e1-28ed-4abb-8af2-66d0f5e0f494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:08.415595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:08.437707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-5598f0e1-28ed-4abb-8af2-66d0f5e0f494 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:08.438942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5598f0e1-28ed-4abb-8af2-66d0f5e0f494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:08.537551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:08.687407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097efc6-d6b3-409f-9f6d-8d8f223b1e6c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:08.689033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1097efc6-d6b3-409f-9f6d-8d8f223b1e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:08.838293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097efc6-d6b3-409f-9f6d-8d8f223b1e6c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:08.850460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:08.952766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe2477-f7f3-4b13-b9c2-1c50abd51052 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:08.953315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe2477-f7f3-4b13-b9c2-1c50abd51052 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:08.953598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe2477-f7f3-4b13-b9c2-1c50abd51052 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:08.953871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe2477-f7f3-4b13-b9c2-1c50abd51052 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:08.955461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-affe2477-f7f3-4b13-b9c2-1c50abd51052 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:09.046066 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe2477-f7f3-4b13-b9c2-1c50abd51052 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:09.047660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe2477-f7f3-4b13-b9c2-1c50abd51052 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:09.065546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 713/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:10.197255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f388e-ede8-4507-bd99-d577c7358057 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:10.197825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f388e-ede8-4507-bd99-d577c7358057 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:10.197825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f388e-ede8-4507-bd99-d577c7358057 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:10.199156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:10.205286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7ba6b-756e-4172-b5de-0d59b57b1ff8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:10.205696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7ba6b-756e-4172-b5de-0d59b57b1ff8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:10.205862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7ba6b-756e-4172-b5de-0d59b57b1ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:10.225468 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:10.281813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7be7ba6b-756e-4172-b5de-0d59b57b1ff8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:10.283015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7be7ba6b-756e-4172-b5de-0d59b57b1ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:10.379088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:10.527344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f388e-ede8-4507-bd99-d577c7358057 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:10.528782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ff9f388e-ede8-4507-bd99-d577c7358057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:10.640638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:10.676752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f388e-ede8-4507-bd99-d577c7358057 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:10.789728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5edff-ca99-41dd-8514-6b7f9b39245a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:10.790337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5edff-ca99-41dd-8514-6b7f9b39245a service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:10.790612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5edff-ca99-41dd-8514-6b7f9b39245a service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:10.790881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5edff-ca99-41dd-8514-6b7f9b39245a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:10.792473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0cf5edff-ca99-41dd-8514-6b7f9b39245a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:10.871793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:10.885023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5edff-ca99-41dd-8514-6b7f9b39245a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:10.886628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5edff-ca99-41dd-8514-6b7f9b39245a service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:10.905588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 711/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:12.017440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dda9ee-02d8-4905-a2a6-b559502dbbb9 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:12.018001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dda9ee-02d8-4905-a2a6-b559502dbbb9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:12.018001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dda9ee-02d8-4905-a2a6-b559502dbbb9 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:12.019242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:12.025572 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2292b7-5064-4ef1-8705-56a2245b08bb None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:12.026018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2292b7-5064-4ef1-8705-56a2245b08bb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:12.026127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2292b7-5064-4ef1-8705-56a2245b08bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:12.103752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-8a2292b7-5064-4ef1-8705-56a2245b08bb None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:12.105299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8a2292b7-5064-4ef1-8705-56a2245b08bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:12.201804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:12.355852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dda9ee-02d8-4905-a2a6-b559502dbbb9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:12.357412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c4dda9ee-02d8-4905-a2a6-b559502dbbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:12.444522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:12.507360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dda9ee-02d8-4905-a2a6-b559502dbbb9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:12.631703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3db7c2-e047-4309-8bfd-18ee6a5b9510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:12.632314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3db7c2-e047-4309-8bfd-18ee6a5b9510 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:12.632593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3db7c2-e047-4309-8bfd-18ee6a5b9510 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:12.632863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3db7c2-e047-4309-8bfd-18ee6a5b9510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:12.634493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ac3db7c2-e047-4309-8bfd-18ee6a5b9510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:12.683929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:12.724798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3db7c2-e047-4309-8bfd-18ee6a5b9510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:12.726418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3db7c2-e047-4309-8bfd-18ee6a5b9510 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:12.745234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 716/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:13.848547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48495f52-25d8-49d9-8320-60ca456465a0 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:13.849105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48495f52-25d8-49d9-8320-60ca456465a0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:13.849105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48495f52-25d8-49d9-8320-60ca456465a0 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:13.850496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:13.857813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5611a-241f-4ed4-8a68-5be8aeaf0c49 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:13.858242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5611a-241f-4ed4-8a68-5be8aeaf0c49 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:13.858353 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5611a-241f-4ed4-8a68-5be8aeaf0c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:13.934483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-24a5611a-241f-4ed4-8a68-5be8aeaf0c49 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:13.935694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-24a5611a-241f-4ed4-8a68-5be8aeaf0c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:14.033424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:14.185674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48495f52-25d8-49d9-8320-60ca456465a0 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:14.187129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-48495f52-25d8-49d9-8320-60ca456465a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:14.235898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:14.333679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48495f52-25d8-49d9-8320-60ca456465a0 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:14.446499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67e608-5893-48a3-8890-8fa9158c09ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:14.447026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67e608-5893-48a3-8890-8fa9158c09ce service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:14.447304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67e608-5893-48a3-8890-8fa9158c09ce service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:14.447597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67e608-5893-48a3-8890-8fa9158c09ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:14.449236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ba67e608-5893-48a3-8890-8fa9158c09ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:14.456139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:14.538683 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67e608-5893-48a3-8890-8fa9158c09ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:14.540297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67e608-5893-48a3-8890-8fa9158c09ce service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:14.558743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 714/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:15.688591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f544a4-3bd4-43d4-af02-cd495d78c109 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:15.689154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f544a4-3bd4-43d4-af02-cd495d78c109 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:15.689255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f544a4-3bd4-43d4-af02-cd495d78c109 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:15.690642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:15.696906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1069fcd-a864-4077-ba9d-7f49e585c771 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:15.697378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1069fcd-a864-4077-ba9d-7f49e585c771 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:15.697520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1069fcd-a864-4077-ba9d-7f49e585c771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:15.774602 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-d1069fcd-a864-4077-ba9d-7f49e585c771 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:15.775811 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d1069fcd-a864-4077-ba9d-7f49e585c771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:15.843512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:15.873626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:16.036358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f544a4-3bd4-43d4-af02-cd495d78c109 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:16.037918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a1f544a4-3bd4-43d4-af02-cd495d78c109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:16.060444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:16.197288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f544a4-3bd4-43d4-af02-cd495d78c109 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:16.284265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:16.310926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da524f-3f51-4200-8a24-b06d79ac3424 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:16.311533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da524f-3f51-4200-8a24-b06d79ac3424 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:16.311812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da524f-3f51-4200-8a24-b06d79ac3424 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:16.312082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da524f-3f51-4200-8a24-b06d79ac3424 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:16.313716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-85da524f-3f51-4200-8a24-b06d79ac3424 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:16.403679 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da524f-3f51-4200-8a24-b06d79ac3424 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:16.405286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da524f-3f51-4200-8a24-b06d79ac3424 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:16.423496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 719/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:17.538924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eecce3-932c-4caa-b15c-afad57966d72 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:17.539332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eecce3-932c-4caa-b15c-afad57966d72 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:17.539483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eecce3-932c-4caa-b15c-afad57966d72 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:17.540853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:17.547491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3f84e-7e7f-4bce-a867-6994442a003a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:17.547923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3f84e-7e7f-4bce-a867-6994442a003a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:17.548109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3f84e-7e7f-4bce-a867-6994442a003a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:17.626161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-64c3f84e-7e7f-4bce-a867-6994442a003a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:17.627379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-64c3f84e-7e7f-4bce-a867-6994442a003a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:17.669799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:17.724273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:17.878081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eecce3-932c-4caa-b15c-afad57966d72 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:17.880246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b1eecce3-932c-4caa-b15c-afad57966d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:18.026370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eecce3-932c-4caa-b15c-afad57966d72 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:18.091064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:18.150792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3421f-7aef-4a46-9629-341c9a84d4cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:18.151322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3421f-7aef-4a46-9629-341c9a84d4cc service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:18.151600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3421f-7aef-4a46-9629-341c9a84d4cc service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:18.151867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3421f-7aef-4a46-9629-341c9a84d4cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:18.153668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-46d3421f-7aef-4a46-9629-341c9a84d4cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:18.244162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3421f-7aef-4a46-9629-341c9a84d4cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:18.245773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3421f-7aef-4a46-9629-341c9a84d4cc service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:18.263876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 717/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:19.379677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680d16-cff1-4063-844f-0ffedb5e706a None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:19.380149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680d16-cff1-4063-844f-0ffedb5e706a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:19.380346 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680d16-cff1-4063-844f-0ffedb5e706a None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:19.381679 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:19.387888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b8367-a668-4029-8dd6-0dc3d937bc61 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:19.388341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b8367-a668-4029-8dd6-0dc3d937bc61 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:19.388447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b8367-a668-4029-8dd6-0dc3d937bc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:19.466326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-750b8367-a668-4029-8dd6-0dc3d937bc61 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:19.467550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-750b8367-a668-4029-8dd6-0dc3d937bc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:19.469619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:19.565786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:19.716053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14680d16-cff1-4063-844f-0ffedb5e706a None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:19.717862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-14680d16-cff1-4063-844f-0ffedb5e706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:19.868988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14680d16-cff1-4063-844f-0ffedb5e706a None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:19.896076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:19.983702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341fcc0-ccfd-4a90-8f5a-fd44de582f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:19.984313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341fcc0-ccfd-4a90-8f5a-fd44de582f8f service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:19.984592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341fcc0-ccfd-4a90-8f5a-fd44de582f8f service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:19.984868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341fcc0-ccfd-4a90-8f5a-fd44de582f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:19.986502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b341fcc0-ccfd-4a90-8f5a-fd44de582f8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:20.078431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341fcc0-ccfd-4a90-8f5a-fd44de582f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:20.080018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341fcc0-ccfd-4a90-8f5a-fd44de582f8f service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:20.100465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 722/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:22.183764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43a80b-8ff5-4fe6-bf74-1f60ceb3d4f1 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:22.184365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43a80b-8ff5-4fe6-bf74-1f60ceb3d4f1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:22.184365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43a80b-8ff5-4fe6-bf74-1f60ceb3d4f1 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:22.185799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:22.191144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722287-0d46-4c64-bc08-5f4fa883556f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:22.192259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722287-0d46-4c64-bc08-5f4fa883556f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:22.192363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a722287-0d46-4c64-bc08-5f4fa883556f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:22.232610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:22.258430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-6a722287-0d46-4c64-bc08-5f4fa883556f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:22.259620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6a722287-0d46-4c64-bc08-5f4fa883556f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:22.357984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:22.642734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43a80b-8ff5-4fe6-bf74-1f60ceb3d4f1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:22.644265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-df43a80b-8ff5-4fe6-bf74-1f60ceb3d4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:22.753790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43a80b-8ff5-4fe6-bf74-1f60ceb3d4f1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:22.787703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:22.827987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d590bd0-e1cf-45b2-b681-8fce63e213f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:22.828589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d590bd0-e1cf-45b2-b681-8fce63e213f6 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:22.828891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d590bd0-e1cf-45b2-b681-8fce63e213f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:22.829199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d590bd0-e1cf-45b2-b681-8fce63e213f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:22.831572 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0d590bd0-e1cf-45b2-b681-8fce63e213f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:22.941980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d590bd0-e1cf-45b2-b681-8fce63e213f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:22.943709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d590bd0-e1cf-45b2-b681-8fce63e213f6 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:22.963335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 720/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:25.276859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba848a69-e6a8-4394-aca1-c103331d6be3 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:25.277484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba848a69-e6a8-4394-aca1-c103331d6be3 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:25.277484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba848a69-e6a8-4394-aca1-c103331d6be3 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:25.278814 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:25.284080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807b375-461b-4f48-9257-618955c1efa2 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:25.284863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807b375-461b-4f48-9257-618955c1efa2 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:25.285705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807b375-461b-4f48-9257-618955c1efa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:25.351250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2807b375-461b-4f48-9257-618955c1efa2 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:25.352456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2807b375-461b-4f48-9257-618955c1efa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:25.366110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:25.454157 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:25.735870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba848a69-e6a8-4394-aca1-c103331d6be3 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:25.737479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ba848a69-e6a8-4394-aca1-c103331d6be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:25.841419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba848a69-e6a8-4394-aca1-c103331d6be3 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:25.854146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:25.916287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba83b2f-19bf-4cc9-b657-dcefc824d008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:25.916808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83b2f-19bf-4cc9-b657-dcefc824d008 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:25.917106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83b2f-19bf-4cc9-b657-dcefc824d008 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:25.917378 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83b2f-19bf-4cc9-b657-dcefc824d008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:25.918938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3ba83b2f-19bf-4cc9-b657-dcefc824d008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:26.028395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba83b2f-19bf-4cc9-b657-dcefc824d008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:26.029845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba83b2f-19bf-4cc9-b657-dcefc824d008 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:26.048169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 725/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:28.591528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bbc97-d0ca-45ef-a69a-9084d888ab7c None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:28.592081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bbc97-d0ca-45ef-a69a-9084d888ab7c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:28.592081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bbc97-d0ca-45ef-a69a-9084d888ab7c None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:28.593474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:28.599213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1de70-ee3e-4625-969d-2a3a90f1cc03 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:28.599627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1de70-ee3e-4625-969d-2a3a90f1cc03 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:28.599760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1de70-ee3e-4625-969d-2a3a90f1cc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:28.648917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:28.666476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-03d1de70-ee3e-4625-969d-2a3a90f1cc03 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:28.668026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-03d1de70-ee3e-4625-969d-2a3a90f1cc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:28.764928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:29.054435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bbc97-d0ca-45ef-a69a-9084d888ab7c None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:29.055971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9a9bbc97-d0ca-45ef-a69a-9084d888ab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:29.167680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bbc97-d0ca-45ef-a69a-9084d888ab7c None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:29.208724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:29.241526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7cf70-723c-45a6-8984-b10abce095a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:29.242359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7cf70-723c-45a6-8984-b10abce095a6 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:29.242819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7cf70-723c-45a6-8984-b10abce095a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:29.243405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7cf70-723c-45a6-8984-b10abce095a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:29.245065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-79d7cf70-723c-45a6-8984-b10abce095a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:29.352184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7cf70-723c-45a6-8984-b10abce095a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:29.353697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7cf70-723c-45a6-8984-b10abce095a6 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:29.372130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 723/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:31.662307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e93536-f59e-4f06-9f11-d31e9e770e2d None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:31.662876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e93536-f59e-4f06-9f11-d31e9e770e2d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:31.662876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e93536-f59e-4f06-9f11-d31e9e770e2d None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:31.664191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:31.669475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652d49e-16e7-4ec8-931f-90f2d3b60a00 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:31.669877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652d49e-16e7-4ec8-931f-90f2d3b60a00 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:31.670037 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652d49e-16e7-4ec8-931f-90f2d3b60a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:31.738394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b652d49e-16e7-4ec8-931f-90f2d3b60a00 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:31.739771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b652d49e-16e7-4ec8-931f-90f2d3b60a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:31.760193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:31.836302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:32.117490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e93536-f59e-4f06-9f11-d31e9e770e2d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:32.119035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-58e93536-f59e-4f06-9f11-d31e9e770e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:32.226123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e93536-f59e-4f06-9f11-d31e9e770e2d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:32.247385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:32.300375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf9352-1f39-4562-bb32-59b92427783e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:32.301586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf9352-1f39-4562-bb32-59b92427783e service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:32.301868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf9352-1f39-4562-bb32-59b92427783e service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:32.302141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf9352-1f39-4562-bb32-59b92427783e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:32.303755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-bfdf9352-1f39-4562-bb32-59b92427783e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:32.413765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf9352-1f39-4562-bb32-59b92427783e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:32.415147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf9352-1f39-4562-bb32-59b92427783e service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:32.433672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 728/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:34.876208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6852f8-c98c-4fa6-840e-d34e6896099b None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:34.876785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6852f8-c98c-4fa6-840e-d34e6896099b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:34.876785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6852f8-c98c-4fa6-840e-d34e6896099b None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:34.878430 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:34.883771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad63c1a-68f0-491b-938f-599e94fa6cea None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:34.884224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad63c1a-68f0-491b-938f-599e94fa6cea None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:34.884333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad63c1a-68f0-491b-938f-599e94fa6cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:34.948326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:34.951032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-5ad63c1a-68f0-491b-938f-599e94fa6cea None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:34.952248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5ad63c1a-68f0-491b-938f-599e94fa6cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:35.052139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:35.345815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6852f8-c98c-4fa6-840e-d34e6896099b None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:35.347904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3e6852f8-c98c-4fa6-840e-d34e6896099b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:35.458068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6852f8-c98c-4fa6-840e-d34e6896099b None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:35.514677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:35.539254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68109dd5-fb18-4475-83d2-b4f783bf6ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:35.539863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68109dd5-fb18-4475-83d2-b4f783bf6ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:35.540141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68109dd5-fb18-4475-83d2-b4f783bf6ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:35.540409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68109dd5-fb18-4475-83d2-b4f783bf6ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:35.542071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-68109dd5-fb18-4475-83d2-b4f783bf6ce6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:35.654768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68109dd5-fb18-4475-83d2-b4f783bf6ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:35.656186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68109dd5-fb18-4475-83d2-b4f783bf6ce6 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:35.674653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 726/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:37.908649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7956d-78f2-4976-9e4f-2859b5f9e3e8 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:37.909198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7956d-78f2-4976-9e4f-2859b5f9e3e8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:37.909198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7956d-78f2-4976-9e4f-2859b5f9e3e8 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:37.911177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:37.916447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab1344-43fd-439c-ae46-6818c78c5bd4 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:37.916890 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab1344-43fd-439c-ae46-6818c78c5bd4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:37.917029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab1344-43fd-439c-ae46-6818c78c5bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:37.983354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e7ab1344-43fd-439c-ae46-6818c78c5bd4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:37.984545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e7ab1344-43fd-439c-ae46-6818c78c5bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:38.014274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:38.088127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:38.379497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7956d-78f2-4976-9e4f-2859b5f9e3e8 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:38.381018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-03f7956d-78f2-4976-9e4f-2859b5f9e3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:38.483973 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7956d-78f2-4976-9e4f-2859b5f9e3e8 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:38.512140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:38.557319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f264b-440a-468e-9831-b7d175ea5092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:38.557903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f264b-440a-468e-9831-b7d175ea5092 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:38.558180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f264b-440a-468e-9831-b7d175ea5092 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:38.558445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f264b-440a-468e-9831-b7d175ea5092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:38.560017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ac0f264b-440a-468e-9831-b7d175ea5092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:38.671798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f264b-440a-468e-9831-b7d175ea5092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:38.673191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f264b-440a-468e-9831-b7d175ea5092 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:38.690953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 731/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:40.900804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ec859-1131-4ff7-beb7-3e534c9f9ad7 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:40.901370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ec859-1131-4ff7-beb7-3e534c9f9ad7 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:40.901370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ec859-1131-4ff7-beb7-3e534c9f9ad7 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:40.902647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:40.907946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e1fd4-397f-4bd4-84c0-71fec74714e2 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:40.908336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e1fd4-397f-4bd4-84c0-71fec74714e2 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:40.908502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e1fd4-397f-4bd4-84c0-71fec74714e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:40.974568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b61e1fd4-397f-4bd4-84c0-71fec74714e2 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:40.975762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b61e1fd4-397f-4bd4-84c0-71fec74714e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:40.984217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:41.072897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:41.368703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ec859-1131-4ff7-beb7-3e534c9f9ad7 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:41.370340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-891ec859-1131-4ff7-beb7-3e534c9f9ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:41.486801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ec859-1131-4ff7-beb7-3e534c9f9ad7 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:41.545883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:41.565556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7ed4a-ff8a-4a54-9dc5-1a74fd254438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:41.566876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7ed4a-ff8a-4a54-9dc5-1a74fd254438 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:41.567374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7ed4a-ff8a-4a54-9dc5-1a74fd254438 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:41.567703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7ed4a-ff8a-4a54-9dc5-1a74fd254438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:41.569857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c1d7ed4a-ff8a-4a54-9dc5-1a74fd254438 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:41.679116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7ed4a-ff8a-4a54-9dc5-1a74fd254438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:41.680567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7ed4a-ff8a-4a54-9dc5-1a74fd254438 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:41.699755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 729/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:44.052734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a13051-20f5-481b-a431-af97ea8621d6 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:44.053282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a13051-20f5-481b-a431-af97ea8621d6 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:44.053282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a13051-20f5-481b-a431-af97ea8621d6 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:44.054651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:44.059892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09cc05-6c55-480b-b0f0-bd6d42515ea5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:44.060368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09cc05-6c55-480b-b0f0-bd6d42515ea5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:44.060466 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09cc05-6c55-480b-b0f0-bd6d42515ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:44.134676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2e09cc05-6c55-480b-b0f0-bd6d42515ea5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:44.135899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2e09cc05-6c55-480b-b0f0-bd6d42515ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:44.174884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:44.240258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:44.531495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a13051-20f5-481b-a431-af97ea8621d6 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:44.533021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-21a13051-20f5-481b-a431-af97ea8621d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:44.636768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a13051-20f5-481b-a431-af97ea8621d6 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:44.676755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:44.713773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e86059-9f12-41f5-9797-19ac0940f3d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:44.714379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86059-9f12-41f5-9797-19ac0940f3d7 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:44.714655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86059-9f12-41f5-9797-19ac0940f3d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:44.714922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86059-9f12-41f5-9797-19ac0940f3d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:44.716499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-52e86059-9f12-41f5-9797-19ac0940f3d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:44.826815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e86059-9f12-41f5-9797-19ac0940f3d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:44.828231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e86059-9f12-41f5-9797-19ac0940f3d7 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:44.847991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 734/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:47.021898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e204006-9f32-4e44-86f1-8242a751681a None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:47.022433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e204006-9f32-4e44-86f1-8242a751681a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:47.022433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e204006-9f32-4e44-86f1-8242a751681a None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:47.023713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:47.028990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fa7c5-158b-4944-94b8-4eed449ba00c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:47.029428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fa7c5-158b-4944-94b8-4eed449ba00c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:47.029602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fa7c5-158b-4944-94b8-4eed449ba00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:47.096040 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-6b9fa7c5-158b-4944-94b8-4eed449ba00c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:47.097306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6b9fa7c5-158b-4944-94b8-4eed449ba00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:47.115310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:47.193428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:47.475312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e204006-9f32-4e44-86f1-8242a751681a None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:47.476848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8e204006-9f32-4e44-86f1-8242a751681a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:47.587601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e204006-9f32-4e44-86f1-8242a751681a None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:47.655760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:47.662369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57804c3c-ec11-442f-ac00-1fa7fa331cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:47.662979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57804c3c-ec11-442f-ac00-1fa7fa331cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:47.663260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57804c3c-ec11-442f-ac00-1fa7fa331cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:47.663537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57804c3c-ec11-442f-ac00-1fa7fa331cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:47.665179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-57804c3c-ec11-442f-ac00-1fa7fa331cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:47.774817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57804c3c-ec11-442f-ac00-1fa7fa331cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:47.776246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57804c3c-ec11-442f-ac00-1fa7fa331cd7 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:47.794599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 732/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:45:50.028198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d512e-bc18-4bb2-9ad6-058056a16a2e None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:50.028731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d512e-bc18-4bb2-9ad6-058056a16a2e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:50.028731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d512e-bc18-4bb2-9ad6-058056a16a2e None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:50.030049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:45:50.036189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c90be-f9d9-4688-8496-c6d454f2047f None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:50.036621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c90be-f9d9-4688-8496-c6d454f2047f None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:50.036742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c90be-f9d9-4688-8496-c6d454f2047f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:50.103819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-5b2c90be-f9d9-4688-8496-c6d454f2047f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:45:50.105332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5b2c90be-f9d9-4688-8496-c6d454f2047f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:50.157169 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:50.206281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:45:50.495674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d512e-bc18-4bb2-9ad6-058056a16a2e None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:45:50.497239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4f5d512e-bc18-4bb2-9ad6-058056a16a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:50.600873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d512e-bc18-4bb2-9ad6-058056a16a2e None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:45:50.648913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:45:50.674458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc638b-9713-416d-9473-9cfd36490970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:45:50.675710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc638b-9713-416d-9473-9cfd36490970 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:45:50.676260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc638b-9713-416d-9473-9cfd36490970 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:45:50.676377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc638b-9713-416d-9473-9cfd36490970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:45:50.677943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-cffc638b-9713-416d-9473-9cfd36490970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:45:50.788430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc638b-9713-416d-9473-9cfd36490970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:45:50.789887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc638b-9713-416d-9473-9cfd36490970 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:45:50.808242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 737/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:46:00.204787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=22 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:00.806023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=66 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:11.688182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ba344-db74-4e96-a06f-4a4974af0d30 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:11.689024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ba344-db74-4e96-a06f-4a4974af0d30 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:11.689350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ba344-db74-4e96-a06f-4a4974af0d30 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:11.691350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 734/1471] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:46:11.696561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fff26c-407b-4c45-b255-594f98987e35 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:11.696978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fff26c-407b-4c45-b255-594f98987e35 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:11.697119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fff26c-407b-4c45-b255-594f98987e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:11.771792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b8fff26c-407b-4c45-b255-594f98987e35 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:46:11.773025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b8fff26c-407b-4c45-b255-594f98987e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:11.871393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 738/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:46:12.177962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37bddb-9070-4bfe-9758-7f5aeda5af25 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:12.178249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37bddb-9070-4bfe-9758-7f5aeda5af25 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:12.178523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37bddb-9070-4bfe-9758-7f5aeda5af25 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:12.179851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 735/1473] 10.222.0.25 () {54 vars in 966 bytes} [Thu Sep 8 18:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:46:12.184877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5319e83-1ae8-4f67-a301-c661ba396554 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:12.185200 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5319e83-1ae8-4f67-a301-c661ba396554 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:12.185482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5319e83-1ae8-4f67-a301-c661ba396554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:12.261120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d5319e83-1ae8-4f67-a301-c661ba396554 None None] MFA Rules not processed for user `c8d10585480f41608bc6935e282f7e27`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:46:12.262319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d5319e83-1ae8-4f67-a301-c661ba396554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:12.282355 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:12.343526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 739/1474] 10.222.0.25 () {58 vars in 1063 bytes} [Thu Sep 8 18:46:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:46:12.351400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37bddb-9070-4bfe-9758-7f5aeda5af25 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:46:12.352902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0f37bddb-9070-4bfe-9758-7f5aeda5af25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:12.462426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37bddb-9070-4bfe-9758-7f5aeda5af25 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:46:12.536201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca8693-1070-4a09-a249-2167bf0f350c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d10585480f41608bc6935e282f7e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:46:12.536813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca8693-1070-4a09-a249-2167bf0f350c service glance] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:12.537138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca8693-1070-4a09-a249-2167bf0f350c service glance] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:12.537446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca8693-1070-4a09-a249-2167bf0f350c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:12.539059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-98ca8693-1070-4a09-a249-2167bf0f350c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:12.593914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:12.650318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca8693-1070-4a09-a249-2167bf0f350c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:46:12.651742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca8693-1070-4a09-a249-2167bf0f350c service glance] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:46:12.670058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 736/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:46:14.954919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6fbb7-fb18-4237-987c-060bbd62b136 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:14.955445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6fbb7-fb18-4237-987c-060bbd62b136 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:14.955445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6fbb7-fb18-4237-987c-060bbd62b136 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:14.957801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 740/1476] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:46:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:46:14.963180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077cec5-855a-4d90-a2fa-6f567c94246d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:14.963603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077cec5-855a-4d90-a2fa-6f567c94246d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:14.963736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077cec5-855a-4d90-a2fa-6f567c94246d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:15.030438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-4077cec5-855a-4d90-a2fa-6f567c94246d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:46:15.031648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4077cec5-855a-4d90-a2fa-6f567c94246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:15.136274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 737/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:46:15.411547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6fbb7-fb18-4237-987c-060bbd62b136 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:46:15.413174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9ff6fbb7-fb18-4237-987c-060bbd62b136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:15.543352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6fbb7-fb18-4237-987c-060bbd62b136 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:46:15.595329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:15.642818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284f1b2-5b7d-43a2-ac3a-43a313c3dbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:46:15.643388 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284f1b2-5b7d-43a2-ac3a-43a313c3dbec admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:15.644255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284f1b2-5b7d-43a2-ac3a-43a313c3dbec admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:15.644578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284f1b2-5b7d-43a2-ac3a-43a313c3dbec admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:15.645302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284f1b2-5b7d-43a2-ac3a-43a313c3dbec admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:46:15.647027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WARNING py.warnings [None req-2284f1b2-5b7d-43a2-ac3a-43a313c3dbec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:46:15.647027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: warnings.warn(msg) Sep 08 18:46:15.647027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: Sep 08 18:46:15.648176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284f1b2-5b7d-43a2-ac3a-43a313c3dbec admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:46:15.667190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 741/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 8 18:46:15 2022] GET /identity/v3/projects => generated 2553 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:46:17.890005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96669d-1b5e-477a-9d47-e844cb441557 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:17.890571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96669d-1b5e-477a-9d47-e844cb441557 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:17.890571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96669d-1b5e-477a-9d47-e844cb441557 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:17.891957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 738/1479] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:46:17.897504 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b629a18-e18d-46a0-80dd-0fdf1122aa72 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:17.897920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b629a18-e18d-46a0-80dd-0fdf1122aa72 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:17.898030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b629a18-e18d-46a0-80dd-0fdf1122aa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:17.963772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1b629a18-e18d-46a0-80dd-0fdf1122aa72 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:46:17.964984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1b629a18-e18d-46a0-80dd-0fdf1122aa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:18.061859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 742/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:46:18.354434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96669d-1b5e-477a-9d47-e844cb441557 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:46:18.356217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1a96669d-1b5e-477a-9d47-e844cb441557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:18.366480 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:18.512206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96669d-1b5e-477a-9d47-e844cb441557 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:46:18.591386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:18.625326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6cc94-0b70-4683-a4ab-4407ff7c4641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:46:18.625931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6cc94-0b70-4683-a4ab-4407ff7c4641 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:18.626207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6cc94-0b70-4683-a4ab-4407ff7c4641 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:18.626476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6cc94-0b70-4683-a4ab-4407ff7c4641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:18.628112 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-aaa6cc94-0b70-4683-a4ab-4407ff7c4641 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:18.716528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6cc94-0b70-4683-a4ab-4407ff7c4641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:46:18.718126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6cc94-0b70-4683-a4ab-4407ff7c4641 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:46:18.742290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 739/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:46:21.081424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266a3d-8cfd-4a97-9293-f57f9e01f4f1 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:21.082005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266a3d-8cfd-4a97-9293-f57f9e01f4f1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:21.082005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266a3d-8cfd-4a97-9293-f57f9e01f4f1 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:21.083331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 743/1482] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:46:21.088741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21214a2c-c34b-45e6-852b-8e3715f54c5d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:21.089233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21214a2c-c34b-45e6-852b-8e3715f54c5d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:21.089341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21214a2c-c34b-45e6-852b-8e3715f54c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:21.155700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-21214a2c-c34b-45e6-852b-8e3715f54c5d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:46:21.156911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-21214a2c-c34b-45e6-852b-8e3715f54c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:21.205024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:21.256433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 740/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:46:21.553877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b266a3d-8cfd-4a97-9293-f57f9e01f4f1 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:46:21.555500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5b266a3d-8cfd-4a97-9293-f57f9e01f4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:21.579492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:21.662135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b266a3d-8cfd-4a97-9293-f57f9e01f4f1 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:46:21.743289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf57a5c-6f46-49ef-819f-6239a9df089a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:46:21.743900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf57a5c-6f46-49ef-819f-6239a9df089a service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:21.744182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf57a5c-6f46-49ef-819f-6239a9df089a service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:21.744457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf57a5c-6f46-49ef-819f-6239a9df089a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:21.746093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-dbf57a5c-6f46-49ef-819f-6239a9df089a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:21.822489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:21.859659 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf57a5c-6f46-49ef-819f-6239a9df089a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:46:21.861115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf57a5c-6f46-49ef-819f-6239a9df089a service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:46:21.880599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 744/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:46:24.075575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2f05e-6e18-49cd-8478-b629f8330417 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:24.076180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2f05e-6e18-49cd-8478-b629f8330417 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:24.076180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2f05e-6e18-49cd-8478-b629f8330417 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:24.077596 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 741/1485] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:46:24.083569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e6516-ed43-4957-9704-f5092b747f1d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:24.084007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e6516-ed43-4957-9704-f5092b747f1d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:24.084117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e6516-ed43-4957-9704-f5092b747f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:24.150142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-eb2e6516-ed43-4957-9704-f5092b747f1d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:46:24.151341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-eb2e6516-ed43-4957-9704-f5092b747f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:24.248648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 745/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:46:24.542727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2f05e-6e18-49cd-8478-b629f8330417 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:46:24.544277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-77b2f05e-6e18-49cd-8478-b629f8330417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:24.650173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2f05e-6e18-49cd-8478-b629f8330417 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:46:24.721178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:24.723223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793cad2b-eec3-44bc-89be-9c358992a76e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:46:24.723821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cad2b-eec3-44bc-89be-9c358992a76e service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:24.724098 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cad2b-eec3-44bc-89be-9c358992a76e service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:24.724397 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cad2b-eec3-44bc-89be-9c358992a76e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:24.726007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-793cad2b-eec3-44bc-89be-9c358992a76e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:24.835481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793cad2b-eec3-44bc-89be-9c358992a76e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:46:24.836822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793cad2b-eec3-44bc-89be-9c358992a76e service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:46:24.854644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 742/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:46:27.207574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80315aa3-10e4-4356-9288-c4badc268ec9 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:27.208147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80315aa3-10e4-4356-9288-c4badc268ec9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:27.208252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80315aa3-10e4-4356-9288-c4badc268ec9 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:27.209583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 746/1488] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:46:27.214830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea73be-c9a7-4b04-896b-551f80965943 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:27.215263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea73be-c9a7-4b04-896b-551f80965943 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:27.215379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea73be-c9a7-4b04-896b-551f80965943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:27.280833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-cdea73be-c9a7-4b04-896b-551f80965943 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:46:27.282061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cdea73be-c9a7-4b04-896b-551f80965943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:27.334178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:27.381010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 743/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:46:27.671655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80315aa3-10e4-4356-9288-c4badc268ec9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:46:27.673230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-80315aa3-10e4-4356-9288-c4badc268ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:27.710490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:27.780974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80315aa3-10e4-4356-9288-c4badc268ec9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:46:27.858296 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e5109-5520-4862-b5c6-76ad718bc608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:46:27.858823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e5109-5520-4862-b5c6-76ad718bc608 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:27.859102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e5109-5520-4862-b5c6-76ad718bc608 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:27.859373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e5109-5520-4862-b5c6-76ad718bc608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:27.860968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-762e5109-5520-4862-b5c6-76ad718bc608 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:27.944267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:27.972417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e5109-5520-4862-b5c6-76ad718bc608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:46:27.973862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e5109-5520-4862-b5c6-76ad718bc608 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:46:27.992771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 747/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:46:30.487711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6daa2-2ff9-4f4b-adff-9fc8eed25604 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:30.488305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6daa2-2ff9-4f4b-adff-9fc8eed25604 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:30.488414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6daa2-2ff9-4f4b-adff-9fc8eed25604 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:30.490304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 744/1491] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 8 18:46:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:46:30.495819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e07a5-5857-4b7a-99e5-0431fd391f2f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:30.496259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e07a5-5857-4b7a-99e5-0431fd391f2f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:30.496383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e07a5-5857-4b7a-99e5-0431fd391f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:30.563233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f40e07a5-5857-4b7a-99e5-0431fd391f2f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:46:30.564448 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f40e07a5-5857-4b7a-99e5-0431fd391f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:30.661274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 748/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 8 18:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:46:30.946716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6daa2-2ff9-4f4b-adff-9fc8eed25604 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:46:30.948256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f7a6daa2-2ff9-4f4b-adff-9fc8eed25604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:31.051102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6daa2-2ff9-4f4b-adff-9fc8eed25604 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:46:31.124561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b0435-9c92-4b4a-94a8-6f09115808ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:46:31.133154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b0435-9c92-4b4a-94a8-6f09115808ae service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:46:31.133154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b0435-9c92-4b4a-94a8-6f09115808ae service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:46:31.133154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b0435-9c92-4b4a-94a8-6f09115808ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:46:31.133154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-562b0435-9c92-4b4a-94a8-6f09115808ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:46:31.136523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:31.236742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b0435-9c92-4b4a-94a8-6f09115808ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:46:31.238148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b0435-9c92-4b4a-94a8-6f09115808ae service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:46:31.255861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 745/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:46:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:46:40.659200 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=82 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:46:41.253767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=50 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:53.009267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7b80-74a3-4f3e-a690-1a89cee09a65 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:53.009858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7b80-74a3-4f3e-a690-1a89cee09a65 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:53.009858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af7b80-74a3-4f3e-a690-1a89cee09a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:53.085615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-59af7b80-74a3-4f3e-a690-1a89cee09a65 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:47:53.086817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-59af7b80-74a3-4f3e-a690-1a89cee09a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:53.182705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 749/1494] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:47:53.190674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192e367-14b9-4454-bc30-7680318b6b41 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:53.191175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192e367-14b9-4454-bc30-7680318b6b41 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:53.191283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192e367-14b9-4454-bc30-7680318b6b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:53.269112 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c192e367-14b9-4454-bc30-7680318b6b41 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:47:53.270747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c192e367-14b9-4454-bc30-7680318b6b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:53.369031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 746/1495] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:47:53.373937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af7b80-74a3-4f3e-a690-1a89cee09a65 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:47:53.375154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-59af7b80-74a3-4f3e-a690-1a89cee09a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:53.386970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af7b80-74a3-4f3e-a690-1a89cee09a65 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:47:53.449151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:53.483843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:47:53.484446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:53.484721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:53.484989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:53.485655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:47:53.485951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:47:53.486988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WARNING py.warnings [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:47:53.486988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: warnings.warn(msg) Sep 08 18:47:53.486988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: Sep 08 18:47:53.487918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:47:53.499226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 750/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:47:53.508045 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192e367-14b9-4454-bc30-7680318b6b41 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:47:53.519579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192e367-14b9-4454-bc30-7680318b6b41 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:47:53.583924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:53.623231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68c166-0a2c-4d2a-aec1-0b7d3d7e1c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:47:53.623838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68c166-0a2c-4d2a-aec1-0b7d3d7e1c4a admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:53.624118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68c166-0a2c-4d2a-aec1-0b7d3d7e1c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:53.624386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68c166-0a2c-4d2a-aec1-0b7d3d7e1c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:53.624998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68c166-0a2c-4d2a-aec1-0b7d3d7e1c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:47:53.626793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: WARNING py.warnings [None req-6c68c166-0a2c-4d2a-aec1-0b7d3d7e1c4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:47:53.626793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: warnings.warn(msg) Sep 08 18:47:53.626793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: Sep 08 18:47:53.628030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68c166-0a2c-4d2a-aec1-0b7d3d7e1c4a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:47:53.744518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 747/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:47:53 2022] POST /identity/v3/projects => generated 375 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:47:53.748695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:47:53.750291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:53.785343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f689c6-dc41-4ff2-898b-1e04d11aa1c5 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:47:53.881263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033ac6d-4c78-4576-9d63-ce3a272de253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:47:53.881870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033ac6d-4c78-4576-9d63-ce3a272de253 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:53.882149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033ac6d-4c78-4576-9d63-ce3a272de253 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:53.882419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033ac6d-4c78-4576-9d63-ce3a272de253 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:53.883038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033ac6d-4c78-4576-9d63-ce3a272de253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:47:53.884035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WARNING py.warnings [None req-a033ac6d-4c78-4576-9d63-ce3a272de253 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:47:53.884035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: warnings.warn(msg) Sep 08 18:47:53.884035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: Sep 08 18:47:53.885261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033ac6d-4c78-4576-9d63-ce3a272de253 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:47:53.966510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 751/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:47:53 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:47:53.970781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68c166-0a2c-4d2a-aec1-0b7d3d7e1c4a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:47:53.972977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6c68c166-0a2c-4d2a-aec1-0b7d3d7e1c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:54.035024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68c166-0a2c-4d2a-aec1-0b7d3d7e1c4a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:47:54.117625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:54.127638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea34813-0919-4692-ae2f-aa8094c6c9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:47:54.128372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea34813-0919-4692-ae2f-aa8094c6c9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:54.128686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea34813-0919-4692-ae2f-aa8094c6c9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:54.128962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea34813-0919-4692-ae2f-aa8094c6c9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:54.129729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea34813-0919-4692-ae2f-aa8094c6c9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:47:54.130738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: WARNING py.warnings [None req-cea34813-0919-4692-ae2f-aa8094c6c9a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:47:54.130738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: warnings.warn(msg) Sep 08 18:47:54.130738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: Sep 08 18:47:54.131624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea34813-0919-4692-ae2f-aa8094c6c9a9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:47:54.145916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 748/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:47:54.149951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033ac6d-4c78-4576-9d63-ce3a272de253 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:47:54.151457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a033ac6d-4c78-4576-9d63-ce3a272de253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:54.163563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033ac6d-4c78-4576-9d63-ce3a272de253 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:47:54.188142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:54.258793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5496df-bd73-48f1-bed2-ee5337c3378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:47:54.259398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5496df-bd73-48f1-bed2-ee5337c3378c admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:54.259676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5496df-bd73-48f1-bed2-ee5337c3378c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:54.259947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5496df-bd73-48f1-bed2-ee5337c3378c admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:54.260582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5496df-bd73-48f1-bed2-ee5337c3378c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:47:54.261617 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WARNING py.warnings [None req-cc5496df-bd73-48f1-bed2-ee5337c3378c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:47:54.261617 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: warnings.warn(msg) Sep 08 18:47:54.261617 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: Sep 08 18:47:54.262506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5496df-bd73-48f1-bed2-ee5337c3378c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:47:54.275225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 752/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:47:54.279529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea34813-0919-4692-ae2f-aa8094c6c9a9 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:47:54.281061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cea34813-0919-4692-ae2f-aa8094c6c9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:54.314917 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea34813-0919-4692-ae2f-aa8094c6c9a9 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:47:54.419958 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49560fe-185b-44fc-a225-1bcb678599f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:47:54.420545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49560fe-185b-44fc-a225-1bcb678599f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:54.420827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49560fe-185b-44fc-a225-1bcb678599f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:54.421130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49560fe-185b-44fc-a225-1bcb678599f3 admin admin] PATH_INFO: `/v3/projects/34d02a4e02ae4ddc9dafbb6d61b07d2d/users/44866cf4210c44bd87b8362deb74f809/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:54.446808 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49560fe-185b-44fc-a225-1bcb678599f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d02a4e02ae4ddc9dafbb6d61b07d2d, user_id=44866cf4210c44bd87b8362deb74f809, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:47:54.447845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: WARNING py.warnings [None req-b49560fe-185b-44fc-a225-1bcb678599f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:47:54.447845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: warnings.warn(msg) Sep 08 18:47:54.447845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: Sep 08 18:47:54.449838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49560fe-185b-44fc-a225-1bcb678599f3 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:47:54.549719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 749/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:47:54 2022] PUT /identity/v3/projects/34d02a4e02ae4ddc9dafbb6d61b07d2d/users/44866cf4210c44bd87b8362deb74f809/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:47:54.562072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385bc7d-fd8f-4e4c-a90f-60456f5d9e68 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:54.562362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385bc7d-fd8f-4e4c-a90f-60456f5d9e68 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:54.562641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385bc7d-fd8f-4e4c-a90f-60456f5d9e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:54.622032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b385bc7d-fd8f-4e4c-a90f-60456f5d9e68 None None] MFA Rules not processed for user `44866cf4210c44bd87b8362deb74f809`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:47:54.623258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b385bc7d-fd8f-4e4c-a90f-60456f5d9e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:54.702141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 753/1502] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:47:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:47:54.715615 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7408dbb-e5fd-4ffe-982f-508f10bddcdc None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:54.715906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7408dbb-e5fd-4ffe-982f-508f10bddcdc None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:54.716195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7408dbb-e5fd-4ffe-982f-508f10bddcdc None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:54.717603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 750/1503] 10.222.0.25 () {54 vars in 964 bytes} [Thu Sep 8 18:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:47:54.722797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd879a-527a-49a6-b4fa-48dabac9a25f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:54.723079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd879a-527a-49a6-b4fa-48dabac9a25f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:54.723352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd879a-527a-49a6-b4fa-48dabac9a25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:54.755899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:54.801995 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-44bd879a-527a-49a6-b4fa-48dabac9a25f None None] MFA Rules not processed for user `605b5d4c0fc9409a94dad22c77c67332`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:47:54.803206 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-44bd879a-527a-49a6-b4fa-48dabac9a25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:54.879916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 754/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Sep 8 18:47:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:47:54.887611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7408dbb-e5fd-4ffe-982f-508f10bddcdc None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:47:54.889167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b7408dbb-e5fd-4ffe-982f-508f10bddcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:54.938728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:54.992208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7408dbb-e5fd-4ffe-982f-508f10bddcdc None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:47:55.072456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f6e44-8c0f-49c7-a47d-006b928b0573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:47:55.073096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6e44-8c0f-49c7-a47d-006b928b0573 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:55.073385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6e44-8c0f-49c7-a47d-006b928b0573 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:55.073663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6e44-8c0f-49c7-a47d-006b928b0573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:55.077257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-610f6e44-8c0f-49c7-a47d-006b928b0573 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:55.178818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f6e44-8c0f-49c7-a47d-006b928b0573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:47:55.180257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f6e44-8c0f-49c7-a47d-006b928b0573 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:47:55.183806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:55.198017 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 751/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:47:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:47:55.244644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61406c7-2f11-4f2f-93ba-555df09c216e None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:55.245072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61406c7-2f11-4f2f-93ba-555df09c216e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:55.245190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61406c7-2f11-4f2f-93ba-555df09c216e None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:55.246461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 755/1506] 10.222.0.25 () {52 vars in 830 bytes} [Thu Sep 8 18:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:47:55.250120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f6e44-8c0f-49c7-a47d-006b928b0573 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:47:55.251345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-610f6e44-8c0f-49c7-a47d-006b928b0573 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:55.350784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f6e44-8c0f-49c7-a47d-006b928b0573 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:47:55.452670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:47:55.453336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:55.453613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:55.453881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] PATH_INFO: `/v3/users/44866cf4210c44bd87b8362deb74f809` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:55.468740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] RBAC: Authorizing `identity:delete_user(user_id=44866cf4210c44bd87b8362deb74f809)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:47:55.469839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: WARNING py.warnings [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:47:55.469839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: warnings.warn(msg) Sep 08 18:47:55.469839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: Sep 08 18:47:55.471048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:47:55.482489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:55.672186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44866cf4210c44bd87b8362deb74f809'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:47:55.672621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44866cf4210c44bd87b8362deb74f809'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:47:55.673343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44866cf4210c44bd87b8362deb74f809'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:47:55.697337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44866cf4210c44bd87b8362deb74f809'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:47:55.714787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44866cf4210c44bd87b8362deb74f809'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:47:55.771974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 752/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:47:55 2022] DELETE /identity/v3/users/44866cf4210c44bd87b8362deb74f809 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:47:55.776603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1944ad8-e4ae-4bfc-8124-b6e0485897f2 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:55.776976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1944ad8-e4ae-4bfc-8124-b6e0485897f2 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:55.777286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1944ad8-e4ae-4bfc-8124-b6e0485897f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:55.836187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f1944ad8-e4ae-4bfc-8124-b6e0485897f2 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:47:55.837440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f1944ad8-e4ae-4bfc-8124-b6e0485897f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:55.914786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:55.927046 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 756/1508] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:47:55.937432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:47:55.938659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:56.038282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b814e-ab10-4c82-8ca9-833cc6f1399b admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:47:56.092474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:47:56.111137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50ca48-f4d6-4b4f-98e0-1c2a45bfaa5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:47:56.111742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ca48-f4d6-4b4f-98e0-1c2a45bfaa5a service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:56.112658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ca48-f4d6-4b4f-98e0-1c2a45bfaa5a service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:56.112876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ca48-f4d6-4b4f-98e0-1c2a45bfaa5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:56.114513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5e50ca48-f4d6-4b4f-98e0-1c2a45bfaa5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:56.223289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50ca48-f4d6-4b4f-98e0-1c2a45bfaa5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:47:56.224685 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50ca48-f4d6-4b4f-98e0-1c2a45bfaa5a service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:47:56.245764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 753/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:47:56.724698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1944ad8-e4ae-4bfc-8124-b6e0485897f2 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:47:56.725922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f1944ad8-e4ae-4bfc-8124-b6e0485897f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:47:56.738959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1944ad8-e4ae-4bfc-8124-b6e0485897f2 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:47:56.852932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:47:56.853600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:47:56.853880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:47:56.854151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] PATH_INFO: `/v3/projects/34d02a4e02ae4ddc9dafbb6d61b07d2d` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:47:56.865297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d02a4e02ae4ddc9dafbb6d61b07d2d)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:47:56.866315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WARNING py.warnings [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:47:56.866315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: warnings.warn(msg) Sep 08 18:47:56.866315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: Sep 08 18:47:56.867510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:47:56.997903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d02a4e02ae4ddc9dafbb6d61b07d2d', 'request_id': 'req-68603867-d447-4995-b693-6a4e66795748'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:47:57.011443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d02a4e02ae4ddc9dafbb6d61b07d2d', 'request_id': 'req-68603867-d447-4995-b693-6a4e66795748'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:47:57.079284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d02a4e02ae4ddc9dafbb6d61b07d2d', 'request_id': 'req-68603867-d447-4995-b693-6a4e66795748'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:47:57.112959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] The token cache is being invalidate because project 34d02a4e02ae4ddc9dafbb6d61b07d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:47:57.113434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:47:57.114145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-68603867-d447-4995-b693-6a4e66795748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:47:57.116263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 757/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:47:56 2022] DELETE /identity/v3/projects/34d02a4e02ae4ddc9dafbb6d61b07d2d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:48:06.243488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=62 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:48:07.109136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:49:24.853505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd04490-37e7-4230-86ec-a24705d3ff29 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:24.854120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd04490-37e7-4230-86ec-a24705d3ff29 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:24.854120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd04490-37e7-4230-86ec-a24705d3ff29 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:24.855399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:49:24.860646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e4936-7d0a-49fa-9b20-68666530df3f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:24.861113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e4936-7d0a-49fa-9b20-68666530df3f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:24.861233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e4936-7d0a-49fa-9b20-68666530df3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:24.944371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-0d6e4936-7d0a-49fa-9b20-68666530df3f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:49:24.945618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0d6e4936-7d0a-49fa-9b20-68666530df3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:25.048526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:49:25.214924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd04490-37e7-4230-86ec-a24705d3ff29 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:49:25.216998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-afd04490-37e7-4230-86ec-a24705d3ff29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:25.335252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd04490-37e7-4230-86ec-a24705d3ff29 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:49:25.340647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5e100-bba1-45f4-bec7-c9bebf0e7a21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:49:25.341274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e100-bba1-45f4-bec7-c9bebf0e7a21 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:25.341554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e100-bba1-45f4-bec7-c9bebf0e7a21 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:25.341823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5e100-bba1-45f4-bec7-c9bebf0e7a21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:25.352006 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5e100-bba1-45f4-bec7-c9bebf0e7a21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:49:25.352998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5e100-bba1-45f4-bec7-c9bebf0e7a21 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:49:25.362711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 755/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:49:26.453648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd532ff3-13b6-4b58-9ebf-1ede057c6193 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:26.454240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd532ff3-13b6-4b58-9ebf-1ede057c6193 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:26.454240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd532ff3-13b6-4b58-9ebf-1ede057c6193 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:26.455539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:49:26.462163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9b2b5-996c-4614-bbf7-07082322893f None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:26.462647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9b2b5-996c-4614-bbf7-07082322893f None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:26.462742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9b2b5-996c-4614-bbf7-07082322893f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:26.483865 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-5af9b2b5-996c-4614-bbf7-07082322893f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:49:26.485110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5af9b2b5-996c-4614-bbf7-07082322893f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:26.493825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:49:26.645718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd532ff3-13b6-4b58-9ebf-1ede057c6193 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:49:26.660479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd532ff3-13b6-4b58-9ebf-1ede057c6193 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:49:26.666054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919fa3b-1045-446c-8fc0-37230824cc31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:49:26.666635 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919fa3b-1045-446c-8fc0-37230824cc31 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:26.666914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919fa3b-1045-446c-8fc0-37230824cc31 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:26.667187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919fa3b-1045-446c-8fc0-37230824cc31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:26.675201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919fa3b-1045-446c-8fc0-37230824cc31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:49:26.677801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919fa3b-1045-446c-8fc0-37230824cc31 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:49:26.691904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 760/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:49:27.021125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce249f01-a58c-4b70-af80-cc84dd0ec853 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:27.021356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce249f01-a58c-4b70-af80-cc84dd0ec853 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:27.021589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce249f01-a58c-4b70-af80-cc84dd0ec853 None None] PATH_INFO: `/v3` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:27.022899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 757/1517] 10.222.0.15 () {54 vars in 925 bytes} [Thu Sep 8 18:49:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:49:27.044264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd51a6-7664-45cd-8869-64820188aca7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:27.044553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd51a6-7664-45cd-8869-64820188aca7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:27.044859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd51a6-7664-45cd-8869-64820188aca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:27.104708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-72bd51a6-7664-45cd-8869-64820188aca7 None None] MFA Rules not processed for user `a074714e365648b88c9d3a0495a17b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:49:27.105951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-72bd51a6-7664-45cd-8869-64820188aca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:27.123231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:49:27.182832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 761/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Sep 8 18:49:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:49:30.993609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844cbf4-3f7f-4c87-9caf-ced12d814b2b None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:30.994128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844cbf4-3f7f-4c87-9caf-ced12d814b2b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:30.994128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844cbf4-3f7f-4c87-9caf-ced12d814b2b None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:30.995488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 758/1519] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 8 18:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:49:31.000494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81392d-50db-4866-bd36-8a29fccbd066 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:31.000910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81392d-50db-4866-bd36-8a29fccbd066 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:31.001026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81392d-50db-4866-bd36-8a29fccbd066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:31.021644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7f81392d-50db-4866-bd36-8a29fccbd066 None None] MFA Rules not processed for user `a074714e365648b88c9d3a0495a17b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:49:31.022827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7f81392d-50db-4866-bd36-8a29fccbd066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:31.031170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 762/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 8 18:49:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:49:31.039009 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844cbf4-3f7f-4c87-9caf-ced12d814b2b None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:49:31.052364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844cbf4-3f7f-4c87-9caf-ced12d814b2b None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:49:31.057216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded765cf-46ad-45b5-b55f-7b0db7192f91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:49:31.057798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded765cf-46ad-45b5-b55f-7b0db7192f91 service placement] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:31.058074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded765cf-46ad-45b5-b55f-7b0db7192f91 service placement] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:31.058344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded765cf-46ad-45b5-b55f-7b0db7192f91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:31.067022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded765cf-46ad-45b5-b55f-7b0db7192f91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:49:31.068586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded765cf-46ad-45b5-b55f-7b0db7192f91 service placement] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:49:31.078000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 759/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:49:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:49:38.228804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632c848-416c-4f90-a1f1-ec0ff55eac20 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:38.229412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632c848-416c-4f90-a1f1-ec0ff55eac20 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:38.229412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632c848-416c-4f90-a1f1-ec0ff55eac20 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:38.231024 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:49:38.239102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab646b5-ce1a-45d7-bdf5-54ff865ffec5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:38.239639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab646b5-ce1a-45d7-bdf5-54ff865ffec5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:38.239743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab646b5-ce1a-45d7-bdf5-54ff865ffec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:38.267432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-dab646b5-ce1a-45d7-bdf5-54ff865ffec5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:49:38.268670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-dab646b5-ce1a-45d7-bdf5-54ff865ffec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:38.278359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:49:38.468333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632c848-416c-4f90-a1f1-ec0ff55eac20 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:49:38.491182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632c848-416c-4f90-a1f1-ec0ff55eac20 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:49:38.498902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e2364-7f0b-49fa-a43b-1866d0a8e7f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:49:38.499522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e2364-7f0b-49fa-a43b-1866d0a8e7f1 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:38.499803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e2364-7f0b-49fa-a43b-1866d0a8e7f1 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:38.500078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e2364-7f0b-49fa-a43b-1866d0a8e7f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:38.509683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e2364-7f0b-49fa-a43b-1866d0a8e7f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:49:38.513255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e2364-7f0b-49fa-a43b-1866d0a8e7f1 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:49:38.532238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 764/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:49:39.677558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d3246-a7e9-45dc-b81d-997d9f6d2731 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:39.678075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d3246-a7e9-45dc-b81d-997d9f6d2731 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:39.678176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d3246-a7e9-45dc-b81d-997d9f6d2731 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:39.679475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 18:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:49:39.685924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d72add-081f-47a9-bd37-c7a06266227a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:39.686378 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d72add-081f-47a9-bd37-c7a06266227a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:39.686479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d72add-081f-47a9-bd37-c7a06266227a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:39.707070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-70d72add-081f-47a9-bd37-c7a06266227a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:49:39.708575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-70d72add-081f-47a9-bd37-c7a06266227a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:39.716088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 18:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:49:39.875004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d3246-a7e9-45dc-b81d-997d9f6d2731 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:49:39.889801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d3246-a7e9-45dc-b81d-997d9f6d2731 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:49:39.896041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f58a3-3f7c-4565-9acb-4f95eea8e930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:49:39.896614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f58a3-3f7c-4565-9acb-4f95eea8e930 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:39.896895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f58a3-3f7c-4565-9acb-4f95eea8e930 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:39.897208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f58a3-3f7c-4565-9acb-4f95eea8e930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:39.905135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f58a3-3f7c-4565-9acb-4f95eea8e930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:49:39.906716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f58a3-3f7c-4565-9acb-4f95eea8e930 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:49:39.916508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 762/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:49:43.922161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfce02-814b-4c30-a133-ffb7ec23e408 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:43.922722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfce02-814b-4c30-a133-ffb7ec23e408 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:43.922722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfce02-814b-4c30-a133-ffb7ec23e408 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:43.924078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:49:43.929783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561bce0-ee47-4145-a526-e90ec2be6ab1 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:43.930208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561bce0-ee47-4145-a526-e90ec2be6ab1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:43.930336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561bce0-ee47-4145-a526-e90ec2be6ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:43.949276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-1561bce0-ee47-4145-a526-e90ec2be6ab1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:49:43.950749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1561bce0-ee47-4145-a526-e90ec2be6ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:43.959090 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:49:44.249930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfce02-814b-4c30-a133-ffb7ec23e408 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:49:44.251639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8dcfce02-814b-4c30-a133-ffb7ec23e408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:44.328653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfce02-814b-4c30-a133-ffb7ec23e408 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:49:44.332920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b63a6-018b-4893-b561-36f8678103b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:49:44.333572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b63a6-018b-4893-b561-36f8678103b3 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:44.333852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b63a6-018b-4893-b561-36f8678103b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:44.334125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b63a6-018b-4893-b561-36f8678103b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:44.342832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b63a6-018b-4893-b561-36f8678103b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:49:44.344232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b63a6-018b-4893-b561-36f8678103b3 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:49:44.354181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 767/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:49:46.519994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa63800-5216-484f-8cfd-3012e71d01e5 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:46.520576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa63800-5216-484f-8cfd-3012e71d01e5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:46.520576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa63800-5216-484f-8cfd-3012e71d01e5 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:46.521954 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:49:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:49:46.527265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1f415-53d7-467e-b708-2647368e7db5 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:46.527670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1f415-53d7-467e-b708-2647368e7db5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:46.527820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1f415-53d7-467e-b708-2647368e7db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:46.548746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-3ca1f415-53d7-467e-b708-2647368e7db5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:49:46.550519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3ca1f415-53d7-467e-b708-2647368e7db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:46.558234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:49:46.849190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa63800-5216-484f-8cfd-3012e71d01e5 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:49:46.854526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=75 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:49:46.860419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa63800-5216-484f-8cfd-3012e71d01e5 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:49:46.864044 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3d353-3d17-4d55-bba4-ef4a3dbc22ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:49:46.867567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3d353-3d17-4d55-bba4-ef4a3dbc22ef service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:46.867567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3d353-3d17-4d55-bba4-ef4a3dbc22ef service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:46.867567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3d353-3d17-4d55-bba4-ef4a3dbc22ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:46.874252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3d353-3d17-4d55-bba4-ef4a3dbc22ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:49:46.875678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3d353-3d17-4d55-bba4-ef4a3dbc22ef service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:49:46.885985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 765/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:49:50.556630 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed71cec-a570-4ba7-b286-76996b9c7648 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:50.557198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed71cec-a570-4ba7-b286-76996b9c7648 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:50.557198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed71cec-a570-4ba7-b286-76996b9c7648 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:50.558578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:49:50.563699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de149d-55d7-4d97-9ba0-0c8d463d84fa None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:50.564096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de149d-55d7-4d97-9ba0-0c8d463d84fa None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:50.564259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de149d-55d7-4d97-9ba0-0c8d463d84fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:50.584530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-02de149d-55d7-4d97-9ba0-0c8d463d84fa None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:49:50.585758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-02de149d-55d7-4d97-9ba0-0c8d463d84fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:50.593778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:49:50.906017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed71cec-a570-4ba7-b286-76996b9c7648 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:49:50.907716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8ed71cec-a570-4ba7-b286-76996b9c7648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:50.973329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:49:50.986533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed71cec-a570-4ba7-b286-76996b9c7648 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:49:50.995240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464eab49-5770-440c-bd25-ab5972229836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:49:50.995240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464eab49-5770-440c-bd25-ab5972229836 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:50.995240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464eab49-5770-440c-bd25-ab5972229836 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:50.995240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464eab49-5770-440c-bd25-ab5972229836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:51.000706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464eab49-5770-440c-bd25-ab5972229836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:49:51.002186 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464eab49-5770-440c-bd25-ab5972229836 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:49:51.012511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 770/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:49:53.167199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21c196-f849-4655-8938-086fb6aa9ea8 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:53.167814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21c196-f849-4655-8938-086fb6aa9ea8 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:53.167814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21c196-f849-4655-8938-086fb6aa9ea8 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:53.169712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 18:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:49:53.175449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb6777-5e4a-4b09-ad4d-f427ceec9558 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:53.176093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb6777-5e4a-4b09-ad4d-f427ceec9558 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:53.176342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb6777-5e4a-4b09-ad4d-f427ceec9558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:53.196345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4aeb6777-5e4a-4b09-ad4d-f427ceec9558 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:49:53.197582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4aeb6777-5e4a-4b09-ad4d-f427ceec9558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:53.205515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 18:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:49:53.507854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21c196-f849-4655-8938-086fb6aa9ea8 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:49:53.509579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1b21c196-f849-4655-8938-086fb6aa9ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:49:53.523823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21c196-f849-4655-8938-086fb6aa9ea8 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:49:53.527349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc16a0a-1d0a-4246-a68e-485567ae6e9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:49:53.527951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc16a0a-1d0a-4246-a68e-485567ae6e9c service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:49:53.528232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc16a0a-1d0a-4246-a68e-485567ae6e9c service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:49:53.528632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc16a0a-1d0a-4246-a68e-485567ae6e9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:49:53.538563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc16a0a-1d0a-4246-a68e-485567ae6e9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:49:53.540002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc16a0a-1d0a-4246-a68e-485567ae6e9c service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:49:53.549619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 768/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:50:03.191244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=11 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:50:03.545357 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=14 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:19.183355 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7dfdf-1350-4c2b-87ca-0f1830d29ed2 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.184022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7dfdf-1350-4c2b-87ca-0f1830d29ed2 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.184022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7dfdf-1350-4c2b-87ca-0f1830d29ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.211590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-98f7dfdf-1350-4c2b-87ca-0f1830d29ed2 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.213284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-98f7dfdf-1350-4c2b-87ca-0f1830d29ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.222262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 772/1540] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.229995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e91735-790d-4b10-8146-1f02d3abf19c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.231929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e91735-790d-4b10-8146-1f02d3abf19c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.231929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e91735-790d-4b10-8146-1f02d3abf19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.233706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a407bbc-3ba9-405e-b15b-c88059fdeb08 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.234268 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a407bbc-3ba9-405e-b15b-c88059fdeb08 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.235030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a407bbc-3ba9-405e-b15b-c88059fdeb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.262507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-60e91735-790d-4b10-8146-1f02d3abf19c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.263744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-60e91735-790d-4b10-8146-1f02d3abf19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.264053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2a407bbc-3ba9-405e-b15b-c88059fdeb08 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.265474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2a407bbc-3ba9-405e-b15b-c88059fdeb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.281528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 769/1541] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.282830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0045536-3a0b-4c95-a99c-4d8e12fb0f73 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.283076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0045536-3a0b-4c95-a99c-4d8e12fb0f73 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.283418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0045536-3a0b-4c95-a99c-4d8e12fb0f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.283638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 773/1542] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.292361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b57b9-9869-445b-94eb-2ea79df9fa36 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.292990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b57b9-9869-445b-94eb-2ea79df9fa36 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.293503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b57b9-9869-445b-94eb-2ea79df9fa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.318077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c0045536-3a0b-4c95-a99c-4d8e12fb0f73 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.319314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c0045536-3a0b-4c95-a99c-4d8e12fb0f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.335077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-348b57b9-9869-445b-94eb-2ea79df9fa36 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.336779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-348b57b9-9869-445b-94eb-2ea79df9fa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.339921 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 770/1543] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.341271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37167a9f-745c-47cb-a60b-07929b80412b None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.341558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37167a9f-745c-47cb-a60b-07929b80412b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.341839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37167a9f-745c-47cb-a60b-07929b80412b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.358743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 774/1544] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.360626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e3384-ac5c-4cb8-9812-2d409416b230 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.361168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e3384-ac5c-4cb8-9812-2d409416b230 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.361646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e3384-ac5c-4cb8-9812-2d409416b230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.380493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-37167a9f-745c-47cb-a60b-07929b80412b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.382067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-37167a9f-745c-47cb-a60b-07929b80412b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.402261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 771/1545] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.402489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37167a9f-745c-47cb-a60b-07929b80412b None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:19.403953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4e8e3384-ac5c-4cb8-9812-2d409416b230 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.405491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4e8e3384-ac5c-4cb8-9812-2d409416b230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.419594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 775/1546] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.420795 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e3384-ac5c-4cb8-9812-2d409416b230 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:19.422853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37167a9f-745c-47cb-a60b-07929b80412b None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:19.436308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e3384-ac5c-4cb8-9812-2d409416b230 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:19.441074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcadaba-5a30-4aba-b617-6a323cb6b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:19.441074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcadaba-5a30-4aba-b617-6a323cb6b266 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.441074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcadaba-5a30-4aba-b617-6a323cb6b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.441074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcadaba-5a30-4aba-b617-6a323cb6b266 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.441588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcadaba-5a30-4aba-b617-6a323cb6b266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:19.441819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcadaba-5a30-4aba-b617-6a323cb6b266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:19.442956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: WARNING py.warnings [None req-0bcadaba-5a30-4aba-b617-6a323cb6b266 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:51:19.442956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: warnings.warn(msg) Sep 08 18:51:19.442956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: Sep 08 18:51:19.443846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcadaba-5a30-4aba-b617-6a323cb6b266 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:19.449549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3cfb8-9301-4e66-a448-a09a3ef775c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:19.450492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3cfb8-9301-4e66-a448-a09a3ef775c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.450987 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3cfb8-9301-4e66-a448-a09a3ef775c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.451462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3cfb8-9301-4e66-a448-a09a3ef775c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.452326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3cfb8-9301-4e66-a448-a09a3ef775c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:19.452844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3cfb8-9301-4e66-a448-a09a3ef775c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:19.455407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3cfb8-9301-4e66-a448-a09a3ef775c4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:19.462946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 772/1547] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:19.464049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcadaba-5a30-4aba-b617-6a323cb6b266 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:19.469932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 776/1548] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:19.471451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8f2a2-e3d0-4714-a75c-051911e465ff None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.471972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8f2a2-e3d0-4714-a75c-051911e465ff None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.472486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8f2a2-e3d0-4714-a75c-051911e465ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.479387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcadaba-5a30-4aba-b617-6a323cb6b266 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:19.484818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b527b3-6a66-4617-85e3-3bd45e03e590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:19.485466 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b527b3-6a66-4617-85e3-3bd45e03e590 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.485749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b527b3-6a66-4617-85e3-3bd45e03e590 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.486032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b527b3-6a66-4617-85e3-3bd45e03e590 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.486680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b527b3-6a66-4617-85e3-3bd45e03e590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:19.486987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b527b3-6a66-4617-85e3-3bd45e03e590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:19.488583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b527b3-6a66-4617-85e3-3bd45e03e590 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:19.493946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d2f8f2a2-e3d0-4714-a75c-051911e465ff None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.495159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d2f8f2a2-e3d0-4714-a75c-051911e465ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.503147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 773/1549] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:19.504694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8e50a-00e3-421d-886f-e18a5f8d34ed None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.504982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8e50a-00e3-421d-886f-e18a5f8d34ed None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.505289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8e50a-00e3-421d-886f-e18a5f8d34ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.505869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 777/1550] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.510320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64e719-a294-4327-b55c-6c70d9ae1712 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.510320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64e719-a294-4327-b55c-6c70d9ae1712 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.510320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64e719-a294-4327-b55c-6c70d9ae1712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.529215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e1d8e50a-00e3-421d-886f-e18a5f8d34ed None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.529458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2b64e719-a294-4327-b55c-6c70d9ae1712 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.530450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e1d8e50a-00e3-421d-886f-e18a5f8d34ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.530634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2b64e719-a294-4327-b55c-6c70d9ae1712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.538548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 778/1551] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.539811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb875dff-6d50-4cf6-8af9-90d47f02d223 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.540093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb875dff-6d50-4cf6-8af9-90d47f02d223 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.540369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb875dff-6d50-4cf6-8af9-90d47f02d223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.541200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 774/1552] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.542704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8e50a-00e3-421d-886f-e18a5f8d34ed None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:19.557388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8e50a-00e3-421d-886f-e18a5f8d34ed None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:19.560088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e51bbc-6cef-4791-a260-fa88d58ffe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:19.560695 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e51bbc-6cef-4791-a260-fa88d58ffe2d admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.560976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e51bbc-6cef-4791-a260-fa88d58ffe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.561106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-cb875dff-6d50-4cf6-8af9-90d47f02d223 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.561279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e51bbc-6cef-4791-a260-fa88d58ffe2d admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.561907 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e51bbc-6cef-4791-a260-fa88d58ffe2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:19.562279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-cb875dff-6d50-4cf6-8af9-90d47f02d223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.563807 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e51bbc-6cef-4791-a260-fa88d58ffe2d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:19.570204 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 779/1553] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.571590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb875dff-6d50-4cf6-8af9-90d47f02d223 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:19.581403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb875dff-6d50-4cf6-8af9-90d47f02d223 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:19.586123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b4133-7794-49c5-9f08-50364c97c208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:19.586772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b4133-7794-49c5-9f08-50364c97c208 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.587057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b4133-7794-49c5-9f08-50364c97c208 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.587335 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b4133-7794-49c5-9f08-50364c97c208 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.587971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b4133-7794-49c5-9f08-50364c97c208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:19.588976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WARNING py.warnings [None req-433b4133-7794-49c5-9f08-50364c97c208 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:51:19.588976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: warnings.warn(msg) Sep 08 18:51:19.588976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: Sep 08 18:51:19.590232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b4133-7794-49c5-9f08-50364c97c208 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:19.701083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 775/1554] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:19.701083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e51bbc-6cef-4791-a260-fa88d58ffe2d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:19.732741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 780/1555] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/projects => generated 391 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:19.732741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b4133-7794-49c5-9f08-50364c97c208 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:19.776241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e51bbc-6cef-4791-a260-fa88d58ffe2d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:19.798967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b4133-7794-49c5-9f08-50364c97c208 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:19.804058 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee34692-609b-486c-b797-dbf16a5efa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:19.804669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee34692-609b-486c-b797-dbf16a5efa4e admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.804942 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee34692-609b-486c-b797-dbf16a5efa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.806076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee34692-609b-486c-b797-dbf16a5efa4e admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.806711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee34692-609b-486c-b797-dbf16a5efa4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:19.808730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee34692-609b-486c-b797-dbf16a5efa4e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:19.834450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51408d99-64e7-4cb3-9e2c-c3d76fe1356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:19.835079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51408d99-64e7-4cb3-9e2c-c3d76fe1356d admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.835341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51408d99-64e7-4cb3-9e2c-c3d76fe1356d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.835612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51408d99-64e7-4cb3-9e2c-c3d76fe1356d admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.836374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51408d99-64e7-4cb3-9e2c-c3d76fe1356d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:19.836679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51408d99-64e7-4cb3-9e2c-c3d76fe1356d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:19.838309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51408d99-64e7-4cb3-9e2c-c3d76fe1356d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:19.849616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 776/1556] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:19.850884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211ed9a-845e-44e7-9a6e-8906f0647fc6 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.851171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211ed9a-845e-44e7-9a6e-8906f0647fc6 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.851447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211ed9a-845e-44e7-9a6e-8906f0647fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.870955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-5211ed9a-845e-44e7-9a6e-8906f0647fc6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.877990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5211ed9a-845e-44e7-9a6e-8906f0647fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.880186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 777/1557] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.881368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacde661-43aa-4381-9e03-d896d0f00fab None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.881651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacde661-43aa-4381-9e03-d896d0f00fab None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.881924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacde661-43aa-4381-9e03-d896d0f00fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.901877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-dacde661-43aa-4381-9e03-d896d0f00fab None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.906673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-dacde661-43aa-4381-9e03-d896d0f00fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.908870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 781/1558] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/projects => generated 367 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:19.910253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1368b-6b45-47b5-91cd-b8004bebc7a4 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.910545 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1368b-6b45-47b5-91cd-b8004bebc7a4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.910824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1368b-6b45-47b5-91cd-b8004bebc7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.914978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 778/1559] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:19.916123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909481b3-1e1d-4d09-9334-9e0106beb408 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:19.916405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909481b3-1e1d-4d09-9334-9e0106beb408 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:19.916678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909481b3-1e1d-4d09-9334-9e0106beb408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:19.925229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:19.939553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c9e1368b-6b45-47b5-91cd-b8004bebc7a4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.939553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c9e1368b-6b45-47b5-91cd-b8004bebc7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:19.939793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-909481b3-1e1d-4d09-9334-9e0106beb408 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:19.939793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-909481b3-1e1d-4d09-9334-9e0106beb408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:20.000695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 782/1560] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:20.000695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1368b-6b45-47b5-91cd-b8004bebc7a4 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.001174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 779/1561] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:20.001324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909481b3-1e1d-4d09-9334-9e0106beb408 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.005388 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:20.010284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1368b-6b45-47b5-91cd-b8004bebc7a4 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.011546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909481b3-1e1d-4d09-9334-9e0106beb408 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.015103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d17319-8a6f-4691-bcd0-789447a38d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.015708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d17319-8a6f-4691-bcd0-789447a38d1f admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.015995 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d17319-8a6f-4691-bcd0-789447a38d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.016269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d17319-8a6f-4691-bcd0-789447a38d1f admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.016894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d17319-8a6f-4691-bcd0-789447a38d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.017077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a2b28-3cd2-4889-847d-575a4b0fcd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.017668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a2b28-3cd2-4889-847d-575a4b0fcd0f admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.017947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a2b28-3cd2-4889-847d-575a4b0fcd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.018219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a2b28-3cd2-4889-847d-575a4b0fcd0f admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.018838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a2b28-3cd2-4889-847d-575a4b0fcd0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.018965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d17319-8a6f-4691-bcd0-789447a38d1f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.019891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: WARNING py.warnings [None req-bb0a2b28-3cd2-4889-847d-575a4b0fcd0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:51:20.019891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: warnings.warn(msg) Sep 08 18:51:20.019891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: Sep 08 18:51:20.021099 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a2b28-3cd2-4889-847d-575a4b0fcd0f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.126560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 783/1562] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:19 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:20.127373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d17319-8a6f-4691-bcd0-789447a38d1f admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.136603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d17319-8a6f-4691-bcd0-789447a38d1f admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.141126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b11cf-dfab-4229-be19-32db12a476dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.141701 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b11cf-dfab-4229-be19-32db12a476dc admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.141979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b11cf-dfab-4229-be19-32db12a476dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.142247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b11cf-dfab-4229-be19-32db12a476dc admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.142855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b11cf-dfab-4229-be19-32db12a476dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.144725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b11cf-dfab-4229-be19-32db12a476dc admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.182897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 780/1563] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:20 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:20.182897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8f77d-f9c1-49f5-b926-da6d78291d86 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.182897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8f77d-f9c1-49f5-b926-da6d78291d86 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.182897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8f77d-f9c1-49f5-b926-da6d78291d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.201189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c2f8f77d-f9c1-49f5-b926-da6d78291d86 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:20.202388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c2f8f77d-f9c1-49f5-b926-da6d78291d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:20.210156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 781/1564] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:20.210925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8f77d-f9c1-49f5-b926-da6d78291d86 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.220348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8f77d-f9c1-49f5-b926-da6d78291d86 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.225016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571cf3f-8b0a-4c5e-854b-c392d4d4db74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.225626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571cf3f-8b0a-4c5e-854b-c392d4d4db74 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.227559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 784/1565] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:20 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:20.228048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571cf3f-8b0a-4c5e-854b-c392d4d4db74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.228048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571cf3f-8b0a-4c5e-854b-c392d4d4db74 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.228231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b11cf-dfab-4229-be19-32db12a476dc admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.228489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571cf3f-8b0a-4c5e-854b-c392d4d4db74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.229748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571cf3f-8b0a-4c5e-854b-c392d4d4db74 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.290019 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b11cf-dfab-4229-be19-32db12a476dc admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.294722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f00e57-0b89-4641-a0c2-16c4b76e17a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.295326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f00e57-0b89-4641-a0c2-16c4b76e17a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.295606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f00e57-0b89-4641-a0c2-16c4b76e17a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.295908 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f00e57-0b89-4641-a0c2-16c4b76e17a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.296569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f00e57-0b89-4641-a0c2-16c4b76e17a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:20.296869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f00e57-0b89-4641-a0c2-16c4b76e17a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.298479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f00e57-0b89-4641-a0c2-16c4b76e17a8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.310397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 785/1566] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:20.311180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f00e57-0b89-4641-a0c2-16c4b76e17a8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.321373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f00e57-0b89-4641-a0c2-16c4b76e17a8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.325773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 782/1567] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:20 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:20.325773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571cf3f-8b0a-4c5e-854b-c392d4d4db74 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.328965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b5f40-8236-4b85-beeb-cd2c26b57f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.329591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b5f40-8236-4b85-beeb-cd2c26b57f49 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.329962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b5f40-8236-4b85-beeb-cd2c26b57f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.330252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b5f40-8236-4b85-beeb-cd2c26b57f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.330959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b5f40-8236-4b85-beeb-cd2c26b57f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:20.331265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b5f40-8236-4b85-beeb-cd2c26b57f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.332835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b5f40-8236-4b85-beeb-cd2c26b57f49 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.335522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571cf3f-8b0a-4c5e-854b-c392d4d4db74 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.340797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603218dd-d315-4c85-9b97-5fdcda2632c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.341512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603218dd-d315-4c85-9b97-5fdcda2632c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.341794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603218dd-d315-4c85-9b97-5fdcda2632c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.342070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603218dd-d315-4c85-9b97-5fdcda2632c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.342864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603218dd-d315-4c85-9b97-5fdcda2632c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:20.343167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603218dd-d315-4c85-9b97-5fdcda2632c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.344757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603218dd-d315-4c85-9b97-5fdcda2632c4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.353482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 786/1568] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:20.353482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b5f40-8236-4b85-beeb-cd2c26b57f49 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.356688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b5f40-8236-4b85-beeb-cd2c26b57f49 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.361958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bac4c0-4017-4a31-9486-0f851943d1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.362554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bac4c0-4017-4a31-9486-0f851943d1da admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.362836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bac4c0-4017-4a31-9486-0f851943d1da admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.363113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bac4c0-4017-4a31-9486-0f851943d1da admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.363812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bac4c0-4017-4a31-9486-0f851943d1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.365578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bac4c0-4017-4a31-9486-0f851943d1da admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.366252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 783/1569] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:20.367155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603218dd-d315-4c85-9b97-5fdcda2632c4 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.380260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 787/1570] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:20.380260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bac4c0-4017-4a31-9486-0f851943d1da admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.386600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603218dd-d315-4c85-9b97-5fdcda2632c4 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.392201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ec67e-b57d-4ed3-9959-df0da74fae6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.392889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ec67e-b57d-4ed3-9959-df0da74fae6b admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.393203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ec67e-b57d-4ed3-9959-df0da74fae6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.393482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ec67e-b57d-4ed3-9959-df0da74fae6b admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.393986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bac4c0-4017-4a31-9486-0f851943d1da admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.394138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ec67e-b57d-4ed3-9959-df0da74fae6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.395761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ec67e-b57d-4ed3-9959-df0da74fae6b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.399299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a99b70-1baa-4f47-ba42-fda9bca3293b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.403465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a99b70-1baa-4f47-ba42-fda9bca3293b admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.403465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a99b70-1baa-4f47-ba42-fda9bca3293b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.403465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a99b70-1baa-4f47-ba42-fda9bca3293b admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.403465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a99b70-1baa-4f47-ba42-fda9bca3293b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:20.403465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a99b70-1baa-4f47-ba42-fda9bca3293b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.403908 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a99b70-1baa-4f47-ba42-fda9bca3293b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.406345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 784/1571] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:20.407266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ec67e-b57d-4ed3-9959-df0da74fae6b admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.416593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 788/1572] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:20.417516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ec67e-b57d-4ed3-9959-df0da74fae6b admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.417644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a99b70-1baa-4f47-ba42-fda9bca3293b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.427244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a99b70-1baa-4f47-ba42-fda9bca3293b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.427643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc90a4-bcb6-4356-8f0a-1fd3bb9574cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.428401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc90a4-bcb6-4356-8f0a-1fd3bb9574cd admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.428684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc90a4-bcb6-4356-8f0a-1fd3bb9574cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.428958 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc90a4-bcb6-4356-8f0a-1fd3bb9574cd admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.429641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc90a4-bcb6-4356-8f0a-1fd3bb9574cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.432072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc90a4-bcb6-4356-8f0a-1fd3bb9574cd admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.432989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3cdb-6608-4ece-9033-6040d910d857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.433665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3cdb-6608-4ece-9033-6040d910d857 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.433945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3cdb-6608-4ece-9033-6040d910d857 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.434219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3cdb-6608-4ece-9033-6040d910d857 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.434842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc3cdb-6608-4ece-9033-6040d910d857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.436731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc3cdb-6608-4ece-9033-6040d910d857 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.563012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 789/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:20 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:20.563012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3cdb-6608-4ece-9033-6040d910d857 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.626152 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3cdb-6608-4ece-9033-6040d910d857 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.630874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a22cf6-762a-445c-a39b-d1476b9f9005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.631482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a22cf6-762a-445c-a39b-d1476b9f9005 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.631763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a22cf6-762a-445c-a39b-d1476b9f9005 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.632037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a22cf6-762a-445c-a39b-d1476b9f9005 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.632676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a22cf6-762a-445c-a39b-d1476b9f9005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.635246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a22cf6-762a-445c-a39b-d1476b9f9005 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.636496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:20.645532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 790/1574] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:20.646428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a22cf6-762a-445c-a39b-d1476b9f9005 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.656848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a22cf6-762a-445c-a39b-d1476b9f9005 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.661425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578dfa30-4294-46b3-94c0-8bf2400b6fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.662145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578dfa30-4294-46b3-94c0-8bf2400b6fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.662424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578dfa30-4294-46b3-94c0-8bf2400b6fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.662696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578dfa30-4294-46b3-94c0-8bf2400b6fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.663303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578dfa30-4294-46b3-94c0-8bf2400b6fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.665315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578dfa30-4294-46b3-94c0-8bf2400b6fe0 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.744323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 785/1575] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:20 2022] POST /identity/v3/users => generated 294 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:20.745254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc90a4-bcb6-4356-8f0a-1fd3bb9574cd admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.754686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc90a4-bcb6-4356-8f0a-1fd3bb9574cd admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.760275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2a562-4db1-464b-b3af-893ba2b15d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.761011 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2a562-4db1-464b-b3af-893ba2b15d73 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.761420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2a562-4db1-464b-b3af-893ba2b15d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.761859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2a562-4db1-464b-b3af-893ba2b15d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.762583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2a562-4db1-464b-b3af-893ba2b15d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.764358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2a562-4db1-464b-b3af-893ba2b15d73 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.777826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 786/1576] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:20.778872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2a562-4db1-464b-b3af-893ba2b15d73 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.781593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 791/1577] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:20 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:20.782511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578dfa30-4294-46b3-94c0-8bf2400b6fe0 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.793407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2a562-4db1-464b-b3af-893ba2b15d73 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.848289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578dfa30-4294-46b3-94c0-8bf2400b6fe0 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.852917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d2251-1981-4de0-8631-e3d3bc7531f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.853554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d2251-1981-4de0-8631-e3d3bc7531f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.853846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d2251-1981-4de0-8631-e3d3bc7531f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.854119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d2251-1981-4de0-8631-e3d3bc7531f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.854753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d2251-1981-4de0-8631-e3d3bc7531f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.855460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f06fc2-e516-47bc-a347-460a5d2a4a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.856103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f06fc2-e516-47bc-a347-460a5d2a4a63 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.856382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f06fc2-e516-47bc-a347-460a5d2a4a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.856490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d2251-1981-4de0-8631-e3d3bc7531f3 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.856667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f06fc2-e516-47bc-a347-460a5d2a4a63 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.857305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f06fc2-e516-47bc-a347-460a5d2a4a63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.859189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f06fc2-e516-47bc-a347-460a5d2a4a63 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.867127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 792/1578] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:20.867952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d2251-1981-4de0-8631-e3d3bc7531f3 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.875708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:20.880188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d2251-1981-4de0-8631-e3d3bc7531f3 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:20.885320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffcb9b-ff74-4e71-8165-8afcbdd9f7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:20.886870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffcb9b-ff74-4e71-8165-8afcbdd9f7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:20.886870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffcb9b-ff74-4e71-8165-8afcbdd9f7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:20.886870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffcb9b-ff74-4e71-8165-8afcbdd9f7d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:20.887176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffcb9b-ff74-4e71-8165-8afcbdd9f7d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:20.889069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffcb9b-ff74-4e71-8165-8afcbdd9f7d0 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:20.970477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 787/1579] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:20 2022] POST /identity/v3/projects => generated 367 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:20.971289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f06fc2-e516-47bc-a347-460a5d2a4a63 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:20.996603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 793/1580] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:20 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:20.997687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffcb9b-ff74-4e71-8165-8afcbdd9f7d0 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.037092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f06fc2-e516-47bc-a347-460a5d2a4a63 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.061903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffcb9b-ff74-4e71-8165-8afcbdd9f7d0 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.067312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8652a5f-fb6b-4f67-bf9f-7a2e781f4267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.068138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8652a5f-fb6b-4f67-bf9f-7a2e781f4267 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.068417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8652a5f-fb6b-4f67-bf9f-7a2e781f4267 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.068695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8652a5f-fb6b-4f67-bf9f-7a2e781f4267 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.069366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8652a5f-fb6b-4f67-bf9f-7a2e781f4267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.071838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8652a5f-fb6b-4f67-bf9f-7a2e781f4267 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.081707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 794/1581] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:21.082511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8652a5f-fb6b-4f67-bf9f-7a2e781f4267 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.091764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8652a5f-fb6b-4f67-bf9f-7a2e781f4267 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.093470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897d79-f2f3-4727-8952-c766c9baac3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.094260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897d79-f2f3-4727-8952-c766c9baac3e admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.094574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897d79-f2f3-4727-8952-c766c9baac3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.094849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897d79-f2f3-4727-8952-c766c9baac3e admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.095475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac897d79-f2f3-4727-8952-c766c9baac3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.096404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f306c89-3f11-471a-bc10-6ebf1d5fb29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.096982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f306c89-3f11-471a-bc10-6ebf1d5fb29d admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.097301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f306c89-3f11-471a-bc10-6ebf1d5fb29d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.097438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac897d79-f2f3-4727-8952-c766c9baac3e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.097598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f306c89-3f11-471a-bc10-6ebf1d5fb29d admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.098198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f306c89-3f11-471a-bc10-6ebf1d5fb29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.099837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f306c89-3f11-471a-bc10-6ebf1d5fb29d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.110335 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 795/1582] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:21.111108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f306c89-3f11-471a-bc10-6ebf1d5fb29d admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.122808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f306c89-3f11-471a-bc10-6ebf1d5fb29d admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.127327 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124aa91-3687-4f27-9d2f-5a5a482a267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.128000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124aa91-3687-4f27-9d2f-5a5a482a267c admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.128274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124aa91-3687-4f27-9d2f-5a5a482a267c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.128547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124aa91-3687-4f27-9d2f-5a5a482a267c admin admin] PATH_INFO: `/v3/projects/b5cd9ee33ccb4212b761bb487384aad6/users/0303d8e14c724505b43abe95a63dcf3c/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.143414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124aa91-3687-4f27-9d2f-5a5a482a267c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5cd9ee33ccb4212b761bb487384aad6, user_id=0303d8e14c724505b43abe95a63dcf3c, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.144421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WARNING py.warnings [None req-a124aa91-3687-4f27-9d2f-5a5a482a267c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:51:21.144421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: warnings.warn(msg) Sep 08 18:51:21.144421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: Sep 08 18:51:21.146390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124aa91-3687-4f27-9d2f-5a5a482a267c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.174909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 788/1583] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:20 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:21.175725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897d79-f2f3-4727-8952-c766c9baac3e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.185802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897d79-f2f3-4727-8952-c766c9baac3e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.190423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc943104-9b90-4b11-8086-b65f6298b991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.191003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc943104-9b90-4b11-8086-b65f6298b991 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.191282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc943104-9b90-4b11-8086-b65f6298b991 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.191556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc943104-9b90-4b11-8086-b65f6298b991 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.192168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc943104-9b90-4b11-8086-b65f6298b991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.194181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc943104-9b90-4b11-8086-b65f6298b991 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.255146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 796/1584] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:21 2022] PUT /identity/v3/projects/b5cd9ee33ccb4212b761bb487384aad6/users/0303d8e14c724505b43abe95a63dcf3c/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:21.256634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124aa91-3687-4f27-9d2f-5a5a482a267c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.275274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:21.292799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 789/1585] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:21 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:21.293970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc943104-9b90-4b11-8086-b65f6298b991 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.303206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc943104-9b90-4b11-8086-b65f6298b991 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.329278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124aa91-3687-4f27-9d2f-5a5a482a267c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.334479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eef7fc-bd9f-4750-b9f7-547d2b4331b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.335624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eef7fc-bd9f-4750-b9f7-547d2b4331b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.335896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eef7fc-bd9f-4750-b9f7-547d2b4331b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.336173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eef7fc-bd9f-4750-b9f7-547d2b4331b1 admin admin] PATH_INFO: `/v3/projects/2b95b79f22a341488f44fc5a840975c9/users/91e852d04f9f464ab8afbaeb9ff355e6/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.354993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eef7fc-bd9f-4750-b9f7-547d2b4331b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b95b79f22a341488f44fc5a840975c9, user_id=91e852d04f9f464ab8afbaeb9ff355e6, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.354993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eef7fc-bd9f-4750-b9f7-547d2b4331b1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.362338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d51bd-13f2-4ce4-b2f2-8f734e92f70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.363007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d51bd-13f2-4ce4-b2f2-8f734e92f70f admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.363225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d51bd-13f2-4ce4-b2f2-8f734e92f70f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.363512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d51bd-13f2-4ce4-b2f2-8f734e92f70f admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.364122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d51bd-13f2-4ce4-b2f2-8f734e92f70f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.366063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d51bd-13f2-4ce4-b2f2-8f734e92f70f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.443812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 797/1586] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:21 2022] PUT /identity/v3/projects/2b95b79f22a341488f44fc5a840975c9/users/91e852d04f9f464ab8afbaeb9ff355e6/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:21.444766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eef7fc-bd9f-4750-b9f7-547d2b4331b1 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.474459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 790/1587] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:21 2022] POST /identity/v3/users => generated 284 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:21.474459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d51bd-13f2-4ce4-b2f2-8f734e92f70f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.480980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:21.485197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d51bd-13f2-4ce4-b2f2-8f734e92f70f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.512326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eef7fc-bd9f-4750-b9f7-547d2b4331b1 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.517118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc7757-d516-4fae-b04f-4c63ce1beb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.517733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc7757-d516-4fae-b04f-4c63ce1beb09 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.518017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc7757-d516-4fae-b04f-4c63ce1beb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.518294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc7757-d516-4fae-b04f-4c63ce1beb09 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.518930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc7757-d516-4fae-b04f-4c63ce1beb09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.520899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc7757-d516-4fae-b04f-4c63ce1beb09 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.544810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f529be-619a-43db-9533-66680d5a3301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.545515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f529be-619a-43db-9533-66680d5a3301 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.545787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f529be-619a-43db-9533-66680d5a3301 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.546067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f529be-619a-43db-9533-66680d5a3301 admin admin] PATH_INFO: `/v3/projects/ebca8e24f4c846bfb316d57be7526fd1/users/fc27de50f6fa494986997a5efdf62af3/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.560432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f529be-619a-43db-9533-66680d5a3301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebca8e24f4c846bfb316d57be7526fd1, user_id=fc27de50f6fa494986997a5efdf62af3, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.563127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f529be-619a-43db-9533-66680d5a3301 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.638889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 798/1588] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:21 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:21.639741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc7757-d516-4fae-b04f-4c63ce1beb09 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.650029 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc7757-d516-4fae-b04f-4c63ce1beb09 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.654573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebc41d-b203-4fbd-a83e-63ef5e38cef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.655156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebc41d-b203-4fbd-a83e-63ef5e38cef2 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.655440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebc41d-b203-4fbd-a83e-63ef5e38cef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.655712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebc41d-b203-4fbd-a83e-63ef5e38cef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.656341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebc41d-b203-4fbd-a83e-63ef5e38cef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.657943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebc41d-b203-4fbd-a83e-63ef5e38cef2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.667971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 799/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:21.668798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebc41d-b203-4fbd-a83e-63ef5e38cef2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.671854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 791/1590] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:21 2022] PUT /identity/v3/projects/ebca8e24f4c846bfb316d57be7526fd1/users/fc27de50f6fa494986997a5efdf62af3/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:21.673310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6547862-215e-46b8-ad85-020932ec7c86 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.673654 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6547862-215e-46b8-ad85-020932ec7c86 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.673996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6547862-215e-46b8-ad85-020932ec7c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.680510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebc41d-b203-4fbd-a83e-63ef5e38cef2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.710618 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f6547862-215e-46b8-ad85-020932ec7c86 None None] MFA Rules not processed for user `0303d8e14c724505b43abe95a63dcf3c`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:21.713022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f6547862-215e-46b8-ad85-020932ec7c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:21.750985 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ffb1c-233f-48a5-baf7-532fc25564cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.751604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ffb1c-233f-48a5-baf7-532fc25564cc admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.753125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ffb1c-233f-48a5-baf7-532fc25564cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.753125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ffb1c-233f-48a5-baf7-532fc25564cc admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.753125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ffb1c-233f-48a5-baf7-532fc25564cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.755343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ffb1c-233f-48a5-baf7-532fc25564cc admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.772542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 800/1591] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:21.772542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ffb1c-233f-48a5-baf7-532fc25564cc admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.779917 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 792/1592] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:21.785120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411505c1-0219-4bc3-a4a0-29b1a8c5c65c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.785120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411505c1-0219-4bc3-a4a0-29b1a8c5c65c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.785120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411505c1-0219-4bc3-a4a0-29b1a8c5c65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.785497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ffb1c-233f-48a5-baf7-532fc25564cc admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.789107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e829a44-9aa0-42b3-9877-2fce60f30540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.790610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e829a44-9aa0-42b3-9877-2fce60f30540 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.791646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e829a44-9aa0-42b3-9877-2fce60f30540 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.791646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e829a44-9aa0-42b3-9877-2fce60f30540 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.791865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e829a44-9aa0-42b3-9877-2fce60f30540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.793439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e829a44-9aa0-42b3-9877-2fce60f30540 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.794894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:21.802701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-411505c1-0219-4bc3-a4a0-29b1a8c5c65c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:21.803889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-411505c1-0219-4bc3-a4a0-29b1a8c5c65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:21.808618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 801/1593] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:21.808618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e829a44-9aa0-42b3-9877-2fce60f30540 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.818241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 793/1594] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:21.818241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411505c1-0219-4bc3-a4a0-29b1a8c5c65c None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.827220 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e829a44-9aa0-42b3-9877-2fce60f30540 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.828594 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411505c1-0219-4bc3-a4a0-29b1a8c5c65c None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.832735 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ecb27-77ee-4e5b-ad35-bc186092d0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.833375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ecb27-77ee-4e5b-ad35-bc186092d0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.833530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbd2b3-dc77-422c-bd37-36b14b6ad782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.833657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ecb27-77ee-4e5b-ad35-bc186092d0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.833918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ecb27-77ee-4e5b-ad35-bc186092d0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.834219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbd2b3-dc77-422c-bd37-36b14b6ad782 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.834506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbd2b3-dc77-422c-bd37-36b14b6ad782 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.834601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ecb27-77ee-4e5b-ad35-bc186092d0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.834779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbd2b3-dc77-422c-bd37-36b14b6ad782 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.835428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fbd2b3-dc77-422c-bd37-36b14b6ad782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.836146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ecb27-77ee-4e5b-ad35-bc186092d0c2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.837669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fbd2b3-dc77-422c-bd37-36b14b6ad782 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.846701 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 802/1595] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:21.847351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ecb27-77ee-4e5b-ad35-bc186092d0c2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.850606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 794/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:21.850606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a248df-5d4b-46ef-8315-f54f28188e10 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.850606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a248df-5d4b-46ef-8315-f54f28188e10 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.850606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a248df-5d4b-46ef-8315-f54f28188e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.857447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ecb27-77ee-4e5b-ad35-bc186092d0c2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.862406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539718d-599e-46d9-ade5-ec989a0dec89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:21.868078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539718d-599e-46d9-ade5-ec989a0dec89 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:21.868078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539718d-599e-46d9-ade5-ec989a0dec89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:21.868078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539718d-599e-46d9-ade5-ec989a0dec89 admin admin] PATH_INFO: `/v3/projects/9f381357ec29474f97a5b353daec21d5/users/2b57c46bc37940718735201095453f16/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:21.878535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539718d-599e-46d9-ade5-ec989a0dec89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f381357ec29474f97a5b353daec21d5, user_id=2b57c46bc37940718735201095453f16, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:21.880537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b4a248df-5d4b-46ef-8315-f54f28188e10 None None] MFA Rules not processed for user `fc27de50f6fa494986997a5efdf62af3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:21.881209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539718d-599e-46d9-ade5-ec989a0dec89 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:21.881840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b4a248df-5d4b-46ef-8315-f54f28188e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:21.931245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:21.948613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 795/1597] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:21.949459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a248df-5d4b-46ef-8315-f54f28188e10 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:21.959809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a248df-5d4b-46ef-8315-f54f28188e10 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:21.960950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 803/1598] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:21 2022] PUT /identity/v3/projects/9f381357ec29474f97a5b353daec21d5/users/2b57c46bc37940718735201095453f16/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:21.962503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539718d-599e-46d9-ade5-ec989a0dec89 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.019799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc99758-a562-4778-980a-5ce79a4b6794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.020611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc99758-a562-4778-980a-5ce79a4b6794 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.020876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc99758-a562-4778-980a-5ce79a4b6794 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.021185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc99758-a562-4778-980a-5ce79a4b6794 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.021832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc99758-a562-4778-980a-5ce79a4b6794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.023493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc99758-a562-4778-980a-5ce79a4b6794 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.025423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539718d-599e-46d9-ade5-ec989a0dec89 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.030057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e71fa8-a790-44c7-a5fc-1a77af614337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.030664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e71fa8-a790-44c7-a5fc-1a77af614337 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.030946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e71fa8-a790-44c7-a5fc-1a77af614337 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.031220 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e71fa8-a790-44c7-a5fc-1a77af614337 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.031857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e71fa8-a790-44c7-a5fc-1a77af614337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.033542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e71fa8-a790-44c7-a5fc-1a77af614337 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.034716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 796/1599] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:22.036085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c29cb8-e88d-4de6-b484-1919e232953f None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.036424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c29cb8-e88d-4de6-b484-1919e232953f None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.036702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c29cb8-e88d-4de6-b484-1919e232953f None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.041255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 797/1600] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 8 18:51:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:51:22.042199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c29cb8-e88d-4de6-b484-1919e232953f None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.052315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 804/1601] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:22.052315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e71fa8-a790-44c7-a5fc-1a77af614337 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.060559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e71fa8-a790-44c7-a5fc-1a77af614337 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.065226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c2c03-9143-45c8-b191-2efb09247cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.065884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c2c03-9143-45c8-b191-2efb09247cdc admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.066071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c2c03-9143-45c8-b191-2efb09247cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.066340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c2c03-9143-45c8-b191-2efb09247cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.066975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c2c03-9143-45c8-b191-2efb09247cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.068553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c2c03-9143-45c8-b191-2efb09247cdc admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.078808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 805/1602] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:22.079597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c2c03-9143-45c8-b191-2efb09247cdc admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.090051 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c2c03-9143-45c8-b191-2efb09247cdc admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.094744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d5f10-a9ff-4ebb-b717-401894395a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.095410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5f10-a9ff-4ebb-b717-401894395a5c admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.095693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5f10-a9ff-4ebb-b717-401894395a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.095964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d5f10-a9ff-4ebb-b717-401894395a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.096590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d5f10-a9ff-4ebb-b717-401894395a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.098212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d5f10-a9ff-4ebb-b717-401894395a5c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.103106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c29cb8-e88d-4de6-b484-1919e232953f None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.106795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69f629-1296-41bb-8264-40db756442a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.107405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f629-1296-41bb-8264-40db756442a5 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.107742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f629-1296-41bb-8264-40db756442a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.107962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f629-1296-41bb-8264-40db756442a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.113093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 806/1603] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:22.113093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d5f10-a9ff-4ebb-b717-401894395a5c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.117905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69f629-1296-41bb-8264-40db756442a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.119377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69f629-1296-41bb-8264-40db756442a5 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.129229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 798/1604] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:22.130281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10630-d380-4b0e-b1f1-d3d9306728fc None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.130575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10630-d380-4b0e-b1f1-d3d9306728fc None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.130897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10630-d380-4b0e-b1f1-d3d9306728fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.153261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b7b10630-d380-4b0e-b1f1-d3d9306728fc None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:22.155211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b7b10630-d380-4b0e-b1f1-d3d9306728fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:22.165102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 799/1605] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:22.166172 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10630-d380-4b0e-b1f1-d3d9306728fc None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.171939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d5f10-a9ff-4ebb-b717-401894395a5c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.176275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10630-d380-4b0e-b1f1-d3d9306728fc None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.178477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb4d33-6a07-403e-b6ea-dad30ca0d43b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.179094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb4d33-6a07-403e-b6ea-dad30ca0d43b service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.179380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb4d33-6a07-403e-b6ea-dad30ca0d43b service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.179655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb4d33-6a07-403e-b6ea-dad30ca0d43b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.181420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cacb27-e11e-4980-ad65-6f9828f0a9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.182080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cacb27-e11e-4980-ad65-6f9828f0a9a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.182481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cacb27-e11e-4980-ad65-6f9828f0a9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.182761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cacb27-e11e-4980-ad65-6f9828f0a9a0 admin admin] PATH_INFO: `/v3/projects/abc8e52cd8ec470b9fbfadb9cf01b9f7/users/58303c18a6314dbeb1994d7813671e2e/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.198514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cacb27-e11e-4980-ad65-6f9828f0a9a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc8e52cd8ec470b9fbfadb9cf01b9f7, user_id=58303c18a6314dbeb1994d7813671e2e, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.201270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cacb27-e11e-4980-ad65-6f9828f0a9a0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.228328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:22.238080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb4d33-6a07-403e-b6ea-dad30ca0d43b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.239513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb4d33-6a07-403e-b6ea-dad30ca0d43b service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.249490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 807/1606] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:22.250566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc10a4-0c98-49d7-848c-5a1d0ecafda8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.250857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc10a4-0c98-49d7-848c-5a1d0ecafda8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.251141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc10a4-0c98-49d7-848c-5a1d0ecafda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.276521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 800/1607] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:22 2022] PUT /identity/v3/projects/abc8e52cd8ec470b9fbfadb9cf01b9f7/users/58303c18a6314dbeb1994d7813671e2e/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:22.276521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cacb27-e11e-4980-ad65-6f9828f0a9a0 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.297384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2fdc10a4-0c98-49d7-848c-5a1d0ecafda8 None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:22.298691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2fdc10a4-0c98-49d7-848c-5a1d0ecafda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:22.372087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:22.376444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cacb27-e11e-4980-ad65-6f9828f0a9a0 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.381252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8db25d-151d-4ae6-a655-526dd49a9d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.382280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8db25d-151d-4ae6-a655-526dd49a9d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.382578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 808/1608] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 8 18:51:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:22.382887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8db25d-151d-4ae6-a655-526dd49a9d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.383394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8db25d-151d-4ae6-a655-526dd49a9d72 admin admin] PATH_INFO: `/v3/projects/6ebd1f96ae5d453caf21c4ef97c41376/users/ffeaf83fa254407abb101e4452436928/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.384407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc10a4-0c98-49d7-848c-5a1d0ecafda8 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.398999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc10a4-0c98-49d7-848c-5a1d0ecafda8 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.400177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8db25d-151d-4ae6-a655-526dd49a9d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ebd1f96ae5d453caf21c4ef97c41376, user_id=ffeaf83fa254407abb101e4452436928, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.403040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8db25d-151d-4ae6-a655-526dd49a9d72 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.403855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f6c41-d836-475d-a653-b62ff55bb894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.404516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f6c41-d836-475d-a653-b62ff55bb894 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.404798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f6c41-d836-475d-a653-b62ff55bb894 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.405096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f6c41-d836-475d-a653-b62ff55bb894 admin admin] PATH_INFO: `/v3/projects/609dc8f22f75429c9d5506ba852411dc/users/ba3778dcb0824695a86b4c8193735a5a/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.423497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f6c41-d836-475d-a653-b62ff55bb894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609dc8f22f75429c9d5506ba852411dc, user_id=ba3778dcb0824695a86b4c8193735a5a, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.426452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f6c41-d836-475d-a653-b62ff55bb894 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.479098 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 801/1609] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:22 2022] PUT /identity/v3/projects/6ebd1f96ae5d453caf21c4ef97c41376/users/ffeaf83fa254407abb101e4452436928/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:22.479098 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8db25d-151d-4ae6-a655-526dd49a9d72 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.526648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 809/1610] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:22 2022] PUT /identity/v3/projects/609dc8f22f75429c9d5506ba852411dc/users/ba3778dcb0824695a86b4c8193735a5a/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:22.539006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f6c41-d836-475d-a653-b62ff55bb894 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.541444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-393f6c41-d836-475d-a653-b62ff55bb894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:22.571751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8db25d-151d-4ae6-a655-526dd49a9d72 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.626594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f6c41-d836-475d-a653-b62ff55bb894 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.630826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaedd36-adc4-41ed-861b-fba25face393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.631556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaedd36-adc4-41ed-861b-fba25face393 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.631908 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaedd36-adc4-41ed-861b-fba25face393 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.632127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaedd36-adc4-41ed-861b-fba25face393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.639489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f192df5-5758-4be1-b03d-dc889d931f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.640292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f192df5-5758-4be1-b03d-dc889d931f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.640640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f192df5-5758-4be1-b03d-dc889d931f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.640942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f192df5-5758-4be1-b03d-dc889d931f15 admin admin] PATH_INFO: `/v3/projects/f865e9e9cb26437daa0a5192f0e739df/users/e0f1b1021b2e48a4af7357ddd67c6daa/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.658391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f192df5-5758-4be1-b03d-dc889d931f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f865e9e9cb26437daa0a5192f0e739df, user_id=e0f1b1021b2e48a4af7357ddd67c6daa, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.661178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f192df5-5758-4be1-b03d-dc889d931f15 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.696919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:22.700458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaedd36-adc4-41ed-861b-fba25face393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.701975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaedd36-adc4-41ed-861b-fba25face393 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.716273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 810/1611] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:22.728526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435d5f2-61ce-47d3-86d8-822d43df1dd4 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.728868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435d5f2-61ce-47d3-86d8-822d43df1dd4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.729184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435d5f2-61ce-47d3-86d8-822d43df1dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.751300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 802/1612] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:22 2022] PUT /identity/v3/projects/f865e9e9cb26437daa0a5192f0e739df/users/e0f1b1021b2e48a4af7357ddd67c6daa/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:22.752643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f192df5-5758-4be1-b03d-dc889d931f15 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:22.759832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7435d5f2-61ce-47d3-86d8-822d43df1dd4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:22.761449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7435d5f2-61ce-47d3-86d8-822d43df1dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:22.850505 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 811/1613] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:22.851877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115c305-28dc-454a-a1a8-456e0cac27a7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.852180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115c305-28dc-454a-a1a8-456e0cac27a7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.852447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115c305-28dc-454a-a1a8-456e0cac27a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.855459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f192df5-5758-4be1-b03d-dc889d931f15 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:22.864016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753e17c-74da-4c30-bea4-428921359daa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:22.868106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753e17c-74da-4c30-bea4-428921359daa service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.868106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753e17c-74da-4c30-bea4-428921359daa service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.868106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753e17c-74da-4c30-bea4-428921359daa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.875757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:22.888829 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2115c305-28dc-454a-a1a8-456e0cac27a7 None None] MFA Rules not processed for user `ffeaf83fa254407abb101e4452436928`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:22.890915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2115c305-28dc-454a-a1a8-456e0cac27a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:22.952116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753e17c-74da-4c30-bea4-428921359daa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:22.953759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753e17c-74da-4c30-bea4-428921359daa service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:22.962751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 803/1614] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:22.970461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd9613-9077-4525-933d-7573842e311a None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.970952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd9613-9077-4525-933d-7573842e311a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.971321 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd9613-9077-4525-933d-7573842e311a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:22.975944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 812/1615] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:22.982592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cd3af-2b9a-44fe-bf69-21599cdc1887 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:22.982592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cd3af-2b9a-44fe-bf69-21599cdc1887 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:22.982592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cd3af-2b9a-44fe-bf69-21599cdc1887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.007300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-02cd9613-9077-4525-933d-7573842e311a None None] MFA Rules not processed for user `ba3778dcb0824695a86b4c8193735a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:23.011026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-02cd9613-9077-4525-933d-7573842e311a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:23.013751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-de2cd3af-2b9a-44fe-bf69-21599cdc1887 None None] MFA Rules not processed for user `e0f1b1021b2e48a4af7357ddd67c6daa`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:23.015202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-de2cd3af-2b9a-44fe-bf69-21599cdc1887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:23.095479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 804/1616] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:23.096448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd9613-9077-4525-933d-7573842e311a None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:23.117816 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 813/1617] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:23.130666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd91a28-5450-437c-8614-2b0491bc142b None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.130666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd91a28-5450-437c-8614-2b0491bc142b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.130666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd91a28-5450-437c-8614-2b0491bc142b None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.130666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 814/1618] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 8 18:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:51:23.130666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd91a28-5450-437c-8614-2b0491bc142b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:23.140676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd91a28-5450-437c-8614-2b0491bc142b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:23.146683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb521e-8a80-4a5e-9d13-c4bdb7aaaad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:23.147699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb521e-8a80-4a5e-9d13-c4bdb7aaaad8 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.148121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb521e-8a80-4a5e-9d13-c4bdb7aaaad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.148506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb521e-8a80-4a5e-9d13-c4bdb7aaaad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.150050 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb521e-8a80-4a5e-9d13-c4bdb7aaaad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:23.155756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb521e-8a80-4a5e-9d13-c4bdb7aaaad8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:23.176562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd9613-9077-4525-933d-7573842e311a None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:23.180600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23556a93-28ab-4a6e-a87e-968a495aea4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:23.181473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23556a93-28ab-4a6e-a87e-968a495aea4c service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.181763 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23556a93-28ab-4a6e-a87e-968a495aea4c service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.182036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23556a93-28ab-4a6e-a87e-968a495aea4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.186469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:23.193904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23556a93-28ab-4a6e-a87e-968a495aea4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:23.195512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23556a93-28ab-4a6e-a87e-968a495aea4c service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:23.210694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 805/1619] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:23.212854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386b418-1249-462e-a987-5e6b12657241 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.215502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386b418-1249-462e-a987-5e6b12657241 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.215502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386b418-1249-462e-a987-5e6b12657241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.237192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-d386b418-1249-462e-a987-5e6b12657241 None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:23.239912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d386b418-1249-462e-a987-5e6b12657241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:23.250383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 806/1620] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 8 18:51:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:23.251822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b54cf8-b3fc-4e88-a946-6e4c0cb82f5c None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.252352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b54cf8-b3fc-4e88-a946-6e4c0cb82f5c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.252731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b54cf8-b3fc-4e88-a946-6e4c0cb82f5c None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.254233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 807/1621] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 8 18:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:51:23.257985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b54cf8-b3fc-4e88-a946-6e4c0cb82f5c None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:23.274741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b54cf8-b3fc-4e88-a946-6e4c0cb82f5c None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:23.281603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d672309-7422-4ec4-8710-dc01b9082a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:23.284050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d672309-7422-4ec4-8710-dc01b9082a0d service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.284050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d672309-7422-4ec4-8710-dc01b9082a0d service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.284050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d672309-7422-4ec4-8710-dc01b9082a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.292252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d672309-7422-4ec4-8710-dc01b9082a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:23.293875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d672309-7422-4ec4-8710-dc01b9082a0d service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:23.304810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 808/1622] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:23.306738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b0d4-4bb6-41ec-abb4-07ef69e606e1 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.307496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b0d4-4bb6-41ec-abb4-07ef69e606e1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.307496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b0d4-4bb6-41ec-abb4-07ef69e606e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.335799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c470b0d4-4bb6-41ec-abb4-07ef69e606e1 None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:23.335799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c470b0d4-4bb6-41ec-abb4-07ef69e606e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:23.340745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 815/1623] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:23 2022] POST /identity/v3/projects => generated 369 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:23.344863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb521e-8a80-4a5e-9d13-c4bdb7aaaad8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:23.349532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 809/1624] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:51:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:23.367956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470b0d4-4bb6-41ec-abb4-07ef69e606e1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:23.385216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:23.473192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb521e-8a80-4a5e-9d13-c4bdb7aaaad8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:23.478959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a91c10-d9fe-4675-a49b-2f5fd6b44cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:23.479691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a91c10-d9fe-4675-a49b-2f5fd6b44cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.480091 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a91c10-d9fe-4675-a49b-2f5fd6b44cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.480610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a91c10-d9fe-4675-a49b-2f5fd6b44cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.481536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a91c10-d9fe-4675-a49b-2f5fd6b44cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:23.483883 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a91c10-d9fe-4675-a49b-2f5fd6b44cf5 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:23.499641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470b0d4-4bb6-41ec-abb4-07ef69e606e1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:23.574185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f8331-3c0f-4700-8dde-3d3b17e8eced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:23.574981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f8331-3c0f-4700-8dde-3d3b17e8eced service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.575281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f8331-3c0f-4700-8dde-3d3b17e8eced service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.575559 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f8331-3c0f-4700-8dde-3d3b17e8eced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.597313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 816/1625] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:23 2022] POST /identity/v3/users => generated 291 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:51:23.598458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a91c10-d9fe-4675-a49b-2f5fd6b44cf5 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:23.623488 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a91c10-d9fe-4675-a49b-2f5fd6b44cf5 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:23.629529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d60310-6ab1-48e3-bc1f-a958461e446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:23.630159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d60310-6ab1-48e3-bc1f-a958461e446e admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.630435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d60310-6ab1-48e3-bc1f-a958461e446e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.630786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d60310-6ab1-48e3-bc1f-a958461e446e admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.631520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d60310-6ab1-48e3-bc1f-a958461e446e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:23.633709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d60310-6ab1-48e3-bc1f-a958461e446e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:23.649594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 817/1626] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:23.649594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d60310-6ab1-48e3-bc1f-a958461e446e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:23.697859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f8331-3c0f-4700-8dde-3d3b17e8eced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:23.699882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f8331-3c0f-4700-8dde-3d3b17e8eced service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:23.716635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 810/1627] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:51:23.717428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f8331-3c0f-4700-8dde-3d3b17e8eced service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:23.729666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f8331-3c0f-4700-8dde-3d3b17e8eced service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:23.738587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28c990-590e-4f1d-9947-c8cd84763e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:23.738587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28c990-590e-4f1d-9947-c8cd84763e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.738587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28c990-590e-4f1d-9947-c8cd84763e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.738587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28c990-590e-4f1d-9947-c8cd84763e09 admin admin] PATH_INFO: `/v3/projects/27f129dbc0ca4a5da78ea1e317b5e718/users/b27e66cff2c845e49d78e6dab365a6ae/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.743035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d60310-6ab1-48e3-bc1f-a958461e446e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:23.751954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1842a7-230e-462d-b4d8-0ece0ef4c714 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:23.752609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1842a7-230e-462d-b4d8-0ece0ef4c714 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.752890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1842a7-230e-462d-b4d8-0ece0ef4c714 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.753244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1842a7-230e-462d-b4d8-0ece0ef4c714 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.759366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28c990-590e-4f1d-9947-c8cd84763e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f129dbc0ca4a5da78ea1e317b5e718, user_id=b27e66cff2c845e49d78e6dab365a6ae, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:23.764711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28c990-590e-4f1d-9947-c8cd84763e09 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:23.835089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1842a7-230e-462d-b4d8-0ece0ef4c714 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:23.836870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1842a7-230e-462d-b4d8-0ece0ef4c714 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:23.854123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 818/1628] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:23.879265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 811/1629] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:23 2022] PUT /identity/v3/projects/27f129dbc0ca4a5da78ea1e317b5e718/users/b27e66cff2c845e49d78e6dab365a6ae/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:23.883779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1842a7-230e-462d-b4d8-0ece0ef4c714 service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:23.886914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb941d8c-5990-4d1c-b31b-efcb4e67765d None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.887402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb941d8c-5990-4d1c-b31b-efcb4e67765d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.887808 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb941d8c-5990-4d1c-b31b-efcb4e67765d None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.889704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 812/1630] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 8 18:51:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:51:23.895083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1842a7-230e-462d-b4d8-0ece0ef4c714 service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:23.895493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a2dd0-0d2a-4fa0-aad9-41af04e66668 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:23.895997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a2dd0-0d2a-4fa0-aad9-41af04e66668 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:23.896503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a2dd0-0d2a-4fa0-aad9-41af04e66668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:23.899798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:23.942077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-970a2dd0-0d2a-4fa0-aad9-41af04e66668 None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:23.943530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-970a2dd0-0d2a-4fa0-aad9-41af04e66668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:24.005774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502803c-b08f-4b3c-b2ef-3bfe10d04b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:24.006488 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502803c-b08f-4b3c-b2ef-3bfe10d04b2e admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:24.006678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502803c-b08f-4b3c-b2ef-3bfe10d04b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:24.006953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502803c-b08f-4b3c-b2ef-3bfe10d04b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:24.007862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502803c-b08f-4b3c-b2ef-3bfe10d04b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:24.010105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502803c-b08f-4b3c-b2ef-3bfe10d04b2e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:24.010624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:24.021766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 819/1631] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:24.026286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502803c-b08f-4b3c-b2ef-3bfe10d04b2e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:24.042659 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502803c-b08f-4b3c-b2ef-3bfe10d04b2e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:24.048337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034067f-7a97-4394-839e-8703f2f7618a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:24.049161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034067f-7a97-4394-839e-8703f2f7618a admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:24.049576 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034067f-7a97-4394-839e-8703f2f7618a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:24.049975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034067f-7a97-4394-839e-8703f2f7618a admin admin] PATH_INFO: `/v3/projects/27f129dbc0ca4a5da78ea1e317b5e718/users/b27e66cff2c845e49d78e6dab365a6ae/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:24.052395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034067f-7a97-4394-839e-8703f2f7618a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f129dbc0ca4a5da78ea1e317b5e718, user_id=b27e66cff2c845e49d78e6dab365a6ae, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:24.056515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034067f-7a97-4394-839e-8703f2f7618a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:24.077421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 813/1632] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:51:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:24.078703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a2dd0-0d2a-4fa0-aad9-41af04e66668 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:24.081092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 820/1633] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:24 2022] PUT /identity/v3/projects/27f129dbc0ca4a5da78ea1e317b5e718/users/b27e66cff2c845e49d78e6dab365a6ae/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:24.091737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303ce86-845a-4463-98b0-1acaa6d3aec5 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:24.092056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303ce86-845a-4463-98b0-1acaa6d3aec5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:24.092359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303ce86-845a-4463-98b0-1acaa6d3aec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:24.133246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4303ce86-845a-4463-98b0-1acaa6d3aec5 None None] MFA Rules not processed for user `b27e66cff2c845e49d78e6dab365a6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:24.134573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4303ce86-845a-4463-98b0-1acaa6d3aec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:24.148225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a2dd0-0d2a-4fa0-aad9-41af04e66668 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:24.231163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70feee9d-b9ea-484f-b784-84577389f9a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:24.231966 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70feee9d-b9ea-484f-b784-84577389f9a7 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:24.232267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70feee9d-b9ea-484f-b784-84577389f9a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:24.233155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70feee9d-b9ea-484f-b784-84577389f9a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:24.250381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 821/1634] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:51:24.329581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70feee9d-b9ea-484f-b784-84577389f9a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:24.331104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70feee9d-b9ea-484f-b784-84577389f9a7 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:24.344113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 814/1635] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:24.609852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303ce86-845a-4463-98b0-1acaa6d3aec5 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:24.625187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303ce86-845a-4463-98b0-1acaa6d3aec5 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:24.630004 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765ee53-9da2-4c35-8e5b-bea39184aa5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:24.630797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ee53-9da2-4c35-8e5b-bea39184aa5b service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:24.631190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ee53-9da2-4c35-8e5b-bea39184aa5b service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:24.632665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ee53-9da2-4c35-8e5b-bea39184aa5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:24.647596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765ee53-9da2-4c35-8e5b-bea39184aa5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:24.650135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765ee53-9da2-4c35-8e5b-bea39184aa5b service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:24.684528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 822/1636] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:26.492973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70feee9d-b9ea-484f-b784-84577389f9a7 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:26.595764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:26.629773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70feee9d-b9ea-484f-b784-84577389f9a7 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:26.636971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:26.637872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:26.638230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:26.643839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] PATH_INFO: `/v3/users/ffeaf83fa254407abb101e4452436928` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:26.643839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffeaf83fa254407abb101e4452436928)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:26.644315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:27.042363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffeaf83fa254407abb101e4452436928'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:27.044603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffeaf83fa254407abb101e4452436928'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:27.045395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffeaf83fa254407abb101e4452436928'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:27.062524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffeaf83fa254407abb101e4452436928'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:27.081713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffeaf83fa254407abb101e4452436928'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:27.216525 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 815/1637] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:26 2022] DELETE /identity/v3/users/ffeaf83fa254407abb101e4452436928 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:27.220996 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8cf70-7c99-4b3e-9edd-0a9d2053c00c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:27.221607 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8cf70-7c99-4b3e-9edd-0a9d2053c00c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:27.222782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f8cf70-7c99-4b3e-9edd-0a9d2053c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:27.287600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-49f8cf70-7c99-4b3e-9edd-0a9d2053c00c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:27.290206 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-49f8cf70-7c99-4b3e-9edd-0a9d2053c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:27.441637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 823/1638] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:27.704240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:27.707486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:27.822553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacafe0-f000-4768-851a-ac3851369318 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:27.843520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e1935-5044-4a56-937e-088c7820c844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:27.853217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1935-5044-4a56-937e-088c7820c844 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:27.853784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1935-5044-4a56-937e-088c7820c844 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:27.854282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1935-5044-4a56-937e-088c7820c844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:27.872596 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e1935-5044-4a56-937e-088c7820c844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:27.877249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e1935-5044-4a56-937e-088c7820c844 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:27.906587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 816/1639] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:29.011069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8cf70-7c99-4b3e-9edd-0a9d2053c00c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:29.015292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-49f8cf70-7c99-4b3e-9edd-0a9d2053c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:29.020249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:29.044094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f8cf70-7c99-4b3e-9edd-0a9d2053c00c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:29.061787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:29.062490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:29.062825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:29.063446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] PATH_INFO: `/v3/projects/6ebd1f96ae5d453caf21c4ef97c41376` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:29.072961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ebd1f96ae5d453caf21c4ef97c41376)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:29.077575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:29.287422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ebd1f96ae5d453caf21c4ef97c41376', 'request_id': 'req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:29.297881 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ebd1f96ae5d453caf21c4ef97c41376', 'request_id': 'req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:29.385307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ebd1f96ae5d453caf21c4ef97c41376', 'request_id': 'req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:29.431531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] The token cache is being invalidate because project 6ebd1f96ae5d453caf21c4ef97c41376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:29.431826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:29.439733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11e80f43-05b0-499f-bd0c-f1209ffa5dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:29.442397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 824/1640] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:29 2022] DELETE /identity/v3/projects/6ebd1f96ae5d453caf21c4ef97c41376 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:29.448630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e811564-833d-4d7a-8d3f-e6719fc0abc6 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:29.449984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e811564-833d-4d7a-8d3f-e6719fc0abc6 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:29.450590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e811564-833d-4d7a-8d3f-e6719fc0abc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:29.530014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2e811564-833d-4d7a-8d3f-e6719fc0abc6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:29.531646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2e811564-833d-4d7a-8d3f-e6719fc0abc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:29.655478 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:29.685193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 817/1641] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:29.692758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d19155-42e0-4873-9558-0dabc6c2799f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:29.696150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d19155-42e0-4873-9558-0dabc6c2799f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:29.696150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d19155-42e0-4873-9558-0dabc6c2799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:29.748569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-41d19155-42e0-4873-9558-0dabc6c2799f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:29.749873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-41d19155-42e0-4873-9558-0dabc6c2799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:29.770627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 825/1642] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:29.774821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e811564-833d-4d7a-8d3f-e6719fc0abc6 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:29.792319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e811564-833d-4d7a-8d3f-e6719fc0abc6 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:29.804129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadfa29-ca65-460f-95ba-41bf9eb22cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:29.804621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadfa29-ca65-460f-95ba-41bf9eb22cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:29.805096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadfa29-ca65-460f-95ba-41bf9eb22cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:29.805528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadfa29-ca65-460f-95ba-41bf9eb22cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:29.806454 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadfa29-ca65-460f-95ba-41bf9eb22cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:29.806763 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadfa29-ca65-460f-95ba-41bf9eb22cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:29.808379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadfa29-ca65-460f-95ba-41bf9eb22cd4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:29.862707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 818/1643] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:29.867413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d19155-42e0-4873-9558-0dabc6c2799f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:29.880707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d19155-42e0-4873-9558-0dabc6c2799f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:29.900024 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a026ec-1159-481a-a64f-9278fdb94781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:29.900673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a026ec-1159-481a-a64f-9278fdb94781 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:29.904255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a026ec-1159-481a-a64f-9278fdb94781 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:29.904255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a026ec-1159-481a-a64f-9278fdb94781 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:29.904255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a026ec-1159-481a-a64f-9278fdb94781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:29.904526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a026ec-1159-481a-a64f-9278fdb94781 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:30.019787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 826/1644] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:29 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:30.024592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadfa29-ca65-460f-95ba-41bf9eb22cd4 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:30.038192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadfa29-ca65-460f-95ba-41bf9eb22cd4 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:30.190302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df8862-5676-4679-af31-67be53f67e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:30.191085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df8862-5676-4679-af31-67be53f67e7e admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:30.191440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df8862-5676-4679-af31-67be53f67e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:30.191790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df8862-5676-4679-af31-67be53f67e7e admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:30.192502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df8862-5676-4679-af31-67be53f67e7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:30.194713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df8862-5676-4679-af31-67be53f67e7e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:30.364928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 819/1645] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:30 2022] POST /identity/v3/users => generated 291 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:30.366302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a026ec-1159-481a-a64f-9278fdb94781 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:30.380450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a026ec-1159-481a-a64f-9278fdb94781 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:30.395425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb108237-c5c8-48a3-b2ee-044104bf2989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:30.396334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108237-c5c8-48a3-b2ee-044104bf2989 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:30.396811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108237-c5c8-48a3-b2ee-044104bf2989 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:30.397307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108237-c5c8-48a3-b2ee-044104bf2989 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:30.398204 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb108237-c5c8-48a3-b2ee-044104bf2989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:30.422999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb108237-c5c8-48a3-b2ee-044104bf2989 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:30.450372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 827/1646] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:30.454111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df8862-5676-4679-af31-67be53f67e7e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:30.472743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df8862-5676-4679-af31-67be53f67e7e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:30.479236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604fd8a-37c5-4ef5-95bc-3a5c3ef62c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:30.480136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604fd8a-37c5-4ef5-95bc-3a5c3ef62c7d admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:30.480680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604fd8a-37c5-4ef5-95bc-3a5c3ef62c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:30.481756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604fd8a-37c5-4ef5-95bc-3a5c3ef62c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:30.482858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604fd8a-37c5-4ef5-95bc-3a5c3ef62c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:30.485278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604fd8a-37c5-4ef5-95bc-3a5c3ef62c7d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:30.498203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 820/1647] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:30.503057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb108237-c5c8-48a3-b2ee-044104bf2989 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:30.520779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb108237-c5c8-48a3-b2ee-044104bf2989 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:30.549266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdde56-6530-427e-8af8-47c1963c5518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:30.550058 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdde56-6530-427e-8af8-47c1963c5518 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:30.557751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdde56-6530-427e-8af8-47c1963c5518 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:30.558380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdde56-6530-427e-8af8-47c1963c5518 admin admin] PATH_INFO: `/v3/projects/ea26513796cd49de8d3e1d8e724c53a5/users/0604b3cfeb664a33868f461f4b792156/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:30.588600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdde56-6530-427e-8af8-47c1963c5518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea26513796cd49de8d3e1d8e724c53a5, user_id=0604b3cfeb664a33868f461f4b792156, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:30.591979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdde56-6530-427e-8af8-47c1963c5518 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:30.725274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 828/1648] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:30 2022] PUT /identity/v3/projects/ea26513796cd49de8d3e1d8e724c53a5/users/0604b3cfeb664a33868f461f4b792156/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:51:30.737790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e1ed-7976-4abd-b435-dd9002d02193 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:30.738365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e1ed-7976-4abd-b435-dd9002d02193 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:30.738871 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e1ed-7976-4abd-b435-dd9002d02193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:30.804324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-3147e1ed-7976-4abd-b435-dd9002d02193 None None] MFA Rules not processed for user `0604b3cfeb664a33868f461f4b792156`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:30.806688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3147e1ed-7976-4abd-b435-dd9002d02193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:30.930842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 821/1649] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:30.935484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdde56-6530-427e-8af8-47c1963c5518 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:30.986253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147e1ed-7976-4abd-b435-dd9002d02193 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:30.988512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3147e1ed-7976-4abd-b435-dd9002d02193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:31.126184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:31.148744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdde56-6530-427e-8af8-47c1963c5518 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:31.156109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147e1ed-7976-4abd-b435-dd9002d02193 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:31.160319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35061eb-7497-40cb-9aab-40042bc11b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:31.160941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35061eb-7497-40cb-9aab-40042bc11b1b admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:31.161757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35061eb-7497-40cb-9aab-40042bc11b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:31.163290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35061eb-7497-40cb-9aab-40042bc11b1b admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:31.165986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35061eb-7497-40cb-9aab-40042bc11b1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:31.168028 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:31.169756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:31.174785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:31.174785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] PATH_INFO: `/v3/users/ba3778dcb0824695a86b4c8193735a5a` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:31.174997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35061eb-7497-40cb-9aab-40042bc11b1b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:31.175074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba3778dcb0824695a86b4c8193735a5a)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:31.180734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:31.295810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 829/1650] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:30 2022] POST /identity/v3/projects => generated 379 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:31.300167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35061eb-7497-40cb-9aab-40042bc11b1b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:31.332674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:31.505555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba3778dcb0824695a86b4c8193735a5a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:31.506313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba3778dcb0824695a86b4c8193735a5a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:31.506964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba3778dcb0824695a86b4c8193735a5a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:31.542807 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba3778dcb0824695a86b4c8193735a5a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:31.544020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35061eb-7497-40cb-9aab-40042bc11b1b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:31.557967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e3470-5299-40c7-aa6e-94322d1b4ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:31.558782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3470-5299-40c7-aa6e-94322d1b4ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:31.559534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba3778dcb0824695a86b4c8193735a5a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:31.559942 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3470-5299-40c7-aa6e-94322d1b4ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:31.560332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3470-5299-40c7-aa6e-94322d1b4ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:31.561131 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e3470-5299-40c7-aa6e-94322d1b4ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:31.567817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e3470-5299-40c7-aa6e-94322d1b4ff4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:31.713287 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 822/1651] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:30 2022] DELETE /identity/v3/users/ba3778dcb0824695a86b4c8193735a5a => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:31.714842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 830/1652] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:31 2022] POST /identity/v3/users => generated 296 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:31.716268 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d6a56-5deb-4265-a003-0b4ac8126d9d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:31.724080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:31.726112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d6a56-5deb-4265-a003-0b4ac8126d9d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:31.727971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d6a56-5deb-4265-a003-0b4ac8126d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:31.728792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:31.790784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b35d6a56-5deb-4265-a003-0b4ac8126d9d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:31.797697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b35d6a56-5deb-4265-a003-0b4ac8126d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:31.850195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b17ed-a001-412e-b255-cba6041c481c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:31.858499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c4983-467d-4778-9c13-39774b5452b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:31.860214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c4983-467d-4778-9c13-39774b5452b2 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:31.863187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c4983-467d-4778-9c13-39774b5452b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:31.863702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c4983-467d-4778-9c13-39774b5452b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:31.864570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c4983-467d-4778-9c13-39774b5452b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:31.866457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c4983-467d-4778-9c13-39774b5452b2 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:31.887958 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 823/1653] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:31.889303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c4983-467d-4778-9c13-39774b5452b2 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:31.892970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d15c4983-467d-4778-9c13-39774b5452b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:31.926474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c4983-467d-4778-9c13-39774b5452b2 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:31.935922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:31.937308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:31.937656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:31.938005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] PATH_INFO: `/v3/users/e0f1b1021b2e48a4af7357ddd67c6daa` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:31.943047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0f1b1021b2e48a4af7357ddd67c6daa)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:31.945999 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:31.996844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 831/1654] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:31.996844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d6a56-5deb-4265-a003-0b4ac8126d9d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:32.020012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d6a56-5deb-4265-a003-0b4ac8126d9d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:32.031615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27959e7-32ff-4a5e-9f94-1267a548d074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:32.033533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27959e7-32ff-4a5e-9f94-1267a548d074 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:32.033820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27959e7-32ff-4a5e-9f94-1267a548d074 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:32.036361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27959e7-32ff-4a5e-9f94-1267a548d074 admin admin] PATH_INFO: `/v3/projects/445f54899a844bd7a7db08c1de9b26db/users/4599045b62ae401fafeb902e17d3ac7a/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:32.075341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27959e7-32ff-4a5e-9f94-1267a548d074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445f54899a844bd7a7db08c1de9b26db, user_id=4599045b62ae401fafeb902e17d3ac7a, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:32.084133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27959e7-32ff-4a5e-9f94-1267a548d074 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:32.187479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0f1b1021b2e48a4af7357ddd67c6daa'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:32.187674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 832/1655] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:31 2022] PUT /identity/v3/projects/445f54899a844bd7a7db08c1de9b26db/users/4599045b62ae401fafeb902e17d3ac7a/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:32.188107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27959e7-32ff-4a5e-9f94-1267a548d074 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:32.191176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0f1b1021b2e48a4af7357ddd67c6daa'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:32.192034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d27959e7-32ff-4a5e-9f94-1267a548d074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:32.192196 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0f1b1021b2e48a4af7357ddd67c6daa'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:32.226121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0f1b1021b2e48a4af7357ddd67c6daa'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:32.246546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0f1b1021b2e48a4af7357ddd67c6daa'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:32.333604 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 824/1656] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:31 2022] DELETE /identity/v3/users/e0f1b1021b2e48a4af7357ddd67c6daa => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:32.334880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:32.342942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:32.412722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27959e7-32ff-4a5e-9f94-1267a548d074 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:32.545939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cc734-ccc6-4708-941d-dff132f0b3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:32.546567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cc734-ccc6-4708-941d-dff132f0b3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:32.546915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cc734-ccc6-4708-941d-dff132f0b3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:32.547112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cc734-ccc6-4708-941d-dff132f0b3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:32.548956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-cc3cc734-ccc6-4708-941d-dff132f0b3f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:32.557307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff87f0-55ee-4ed3-b920-1758069526a7 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:32.573804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae093df-364e-47b0-8270-d8382476b475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:32.574486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae093df-364e-47b0-8270-d8382476b475 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:32.574926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae093df-364e-47b0-8270-d8382476b475 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:32.575402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae093df-364e-47b0-8270-d8382476b475 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:32.575609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3cc734-ccc6-4708-941d-dff132f0b3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:32.576378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae093df-364e-47b0-8270-d8382476b475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:32.577099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3cc734-ccc6-4708-941d-dff132f0b3f4 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:32.581794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae093df-364e-47b0-8270-d8382476b475 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:32.598278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 833/1657] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:32.599589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33f3da-0d98-48be-8728-f98bdf4be498 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:32.599891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33f3da-0d98-48be-8728-f98bdf4be498 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:32.600153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33f3da-0d98-48be-8728-f98bdf4be498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:32.610328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 825/1658] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:32.614703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae093df-364e-47b0-8270-d8382476b475 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:32.656356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae093df-364e-47b0-8270-d8382476b475 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:32.673245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6c34d-c9da-4fce-a24b-87b53d29f35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:32.674489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6c34d-c9da-4fce-a24b-87b53d29f35d admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:32.674887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6c34d-c9da-4fce-a24b-87b53d29f35d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:32.675276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6c34d-c9da-4fce-a24b-87b53d29f35d admin admin] PATH_INFO: `/v3/projects/445f54899a844bd7a7db08c1de9b26db/users/4599045b62ae401fafeb902e17d3ac7a/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:32.681335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6c34d-c9da-4fce-a24b-87b53d29f35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=445f54899a844bd7a7db08c1de9b26db, user_id=4599045b62ae401fafeb902e17d3ac7a, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:32.694945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-6f33f3da-0d98-48be-8728-f98bdf4be498 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:32.697180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6f33f3da-0d98-48be-8728-f98bdf4be498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:32.697705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6c34d-c9da-4fce-a24b-87b53d29f35d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:32.702430 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:32.734027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 834/1659] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:32.753723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 826/1660] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:32 2022] PUT /identity/v3/projects/445f54899a844bd7a7db08c1de9b26db/users/4599045b62ae401fafeb902e17d3ac7a/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:51:32.779610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f7dd-805b-4e96-b681-656c91a31e08 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:32.779610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f7dd-805b-4e96-b681-656c91a31e08 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:32.780138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f7dd-805b-4e96-b681-656c91a31e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:32.799007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6c34d-c9da-4fce-a24b-87b53d29f35d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:32.800923 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-31c6c34d-c9da-4fce-a24b-87b53d29f35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:32.882704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e543f7dd-805b-4e96-b681-656c91a31e08 None None] MFA Rules not processed for user `4599045b62ae401fafeb902e17d3ac7a`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:32.889554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e543f7dd-805b-4e96-b681-656c91a31e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:33.033681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6c34d-c9da-4fce-a24b-87b53d29f35d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:33.049312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ec2de-db15-443f-9943-e7a5d2e7d036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:33.050075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec2de-db15-443f-9943-e7a5d2e7d036 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:33.050228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec2de-db15-443f-9943-e7a5d2e7d036 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:33.050498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec2de-db15-443f-9943-e7a5d2e7d036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:33.162798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 835/1661] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:33.227074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543f7dd-805b-4e96-b681-656c91a31e08 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:33.228851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e543f7dd-805b-4e96-b681-656c91a31e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:33.261260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ec2de-db15-443f-9943-e7a5d2e7d036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:33.262731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ec2de-db15-443f-9943-e7a5d2e7d036 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:33.284754 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 827/1662] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:33.403707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543f7dd-805b-4e96-b681-656c91a31e08 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:33.422079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797220b-df7f-4d0c-a4aa-aa412be12027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:33.422079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797220b-df7f-4d0c-a4aa-aa412be12027 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:33.422079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797220b-df7f-4d0c-a4aa-aa412be12027 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:33.422079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797220b-df7f-4d0c-a4aa-aa412be12027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:33.443952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797220b-df7f-4d0c-a4aa-aa412be12027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:33.449189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797220b-df7f-4d0c-a4aa-aa412be12027 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:33.465921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 836/1663] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:33.604485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:33.841038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ec2de-db15-443f-9943-e7a5d2e7d036 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:33.853152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ec2de-db15-443f-9943-e7a5d2e7d036 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:33.869679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766b84-aa74-4e30-afde-df26de9358f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:33.870639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766b84-aa74-4e30-afde-df26de9358f1 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:33.871077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766b84-aa74-4e30-afde-df26de9358f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:33.872729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766b84-aa74-4e30-afde-df26de9358f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:33.881937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-42766b84-aa74-4e30-afde-df26de9358f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:34.096367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42766b84-aa74-4e30-afde-df26de9358f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:34.098003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42766b84-aa74-4e30-afde-df26de9358f1 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:34.128790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 828/1664] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:34.272653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:34.403980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19010530-bb58-4b72-a4c9-a3fbd2c6da2a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:34.404964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19010530-bb58-4b72-a4c9-a3fbd2c6da2a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:34.405470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19010530-bb58-4b72-a4c9-a3fbd2c6da2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:34.518602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-19010530-bb58-4b72-a4c9-a3fbd2c6da2a None None] MFA Rules not processed for user `91e852d04f9f464ab8afbaeb9ff355e6`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:34.518602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-19010530-bb58-4b72-a4c9-a3fbd2c6da2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:34.663288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766b84-aa74-4e30-afde-df26de9358f1 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:34.665775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-42766b84-aa74-4e30-afde-df26de9358f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:34.686810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:34.696671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766b84-aa74-4e30-afde-df26de9358f1 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:34.707602 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:34.708264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:34.708548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:34.708829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] PATH_INFO: `/v3/projects/f865e9e9cb26437daa0a5192f0e739df` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:34.710351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f865e9e9cb26437daa0a5192f0e739df)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:34.712767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: WARNING py.warnings [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:51:34.712767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: warnings.warn(msg) Sep 08 18:51:34.712767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: Sep 08 18:51:34.717730 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:34.749276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 837/1665] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:34.751332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0befc-4a74-43aa-8c52-e389b4cbb0ba None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:34.751818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0befc-4a74-43aa-8c52-e389b4cbb0ba None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:34.752200 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0befc-4a74-43aa-8c52-e389b4cbb0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:34.799041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7ad0befc-4a74-43aa-8c52-e389b4cbb0ba None None] MFA Rules not processed for user `91e852d04f9f464ab8afbaeb9ff355e6`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:34.808686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7ad0befc-4a74-43aa-8c52-e389b4cbb0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:34.827049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 838/1666] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:51:34.838636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0befc-4a74-43aa-8c52-e389b4cbb0ba None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:34.840299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7ad0befc-4a74-43aa-8c52-e389b4cbb0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:34.935934 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f865e9e9cb26437daa0a5192f0e739df', 'request_id': 'req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:34.966794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f865e9e9cb26437daa0a5192f0e739df', 'request_id': 'req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:34.976019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f865e9e9cb26437daa0a5192f0e739df', 'request_id': 'req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:35.059157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0befc-4a74-43aa-8c52-e389b4cbb0ba None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:35.155770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] The token cache is being invalidate because project f865e9e9cb26437daa0a5192f0e739df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:35.158841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:35.158841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:35.161236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 829/1667] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:34 2022] DELETE /identity/v3/projects/f865e9e9cb26437daa0a5192f0e739df => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:35.162562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:35.168205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:35.269564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231938f2-2a6b-4480-88e8-2b8d428759cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:35.270686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231938f2-2a6b-4480-88e8-2b8d428759cf service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:35.270786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231938f2-2a6b-4480-88e8-2b8d428759cf service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:35.271160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231938f2-2a6b-4480-88e8-2b8d428759cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:35.280306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-231938f2-2a6b-4480-88e8-2b8d428759cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:35.347310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0c053-872c-43c1-9e86-ecfbe0ae74bb admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:35.378064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231938f2-2a6b-4480-88e8-2b8d428759cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:35.380326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231938f2-2a6b-4480-88e8-2b8d428759cf service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:35.383223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:35.384014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:35.384303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:35.384587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] PATH_INFO: `/v3/users/0303d8e14c724505b43abe95a63dcf3c` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:35.385889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0303d8e14c724505b43abe95a63dcf3c)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:35.388053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:35.394891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 839/1668] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:51:35.396655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4089028-e2ad-45bc-8e4f-eac8923f922d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:35.397684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4089028-e2ad-45bc-8e4f-eac8923f922d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:35.398050 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4089028-e2ad-45bc-8e4f-eac8923f922d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:35.430950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b4089028-e2ad-45bc-8e4f-eac8923f922d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:35.432413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b4089028-e2ad-45bc-8e4f-eac8923f922d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:35.442583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 840/1669] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:35.454363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a9bb2-d11e-4e2a-bb13-a605515ab964 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:35.454649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a9bb2-d11e-4e2a-bb13-a605515ab964 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:35.454928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a9bb2-d11e-4e2a-bb13-a605515ab964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:35.477059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:35.511326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-825a9bb2-d11e-4e2a-bb13-a605515ab964 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:35.514631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-825a9bb2-d11e-4e2a-bb13-a605515ab964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:35.581917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 841/1670] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:51:35.586666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a9bb2-d11e-4e2a-bb13-a605515ab964 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:35.631380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a9bb2-d11e-4e2a-bb13-a605515ab964 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:35.638319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4167524-d537-4588-9419-00e96d026e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:35.641020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4167524-d537-4588-9419-00e96d026e18 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:35.641556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4167524-d537-4588-9419-00e96d026e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:35.642841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4167524-d537-4588-9419-00e96d026e18 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:35.643796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4167524-d537-4588-9419-00e96d026e18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:35.644320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4167524-d537-4588-9419-00e96d026e18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:35.655052 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4167524-d537-4588-9419-00e96d026e18 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:35.679129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 842/1671] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:35.683725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4167524-d537-4588-9419-00e96d026e18 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:35.691659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0303d8e14c724505b43abe95a63dcf3c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:35.692774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0303d8e14c724505b43abe95a63dcf3c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:35.695152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0303d8e14c724505b43abe95a63dcf3c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:35.704865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4167524-d537-4588-9419-00e96d026e18 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:35.721407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0478f8-8a4a-47ef-99f4-cfe2679744b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:35.723048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0478f8-8a4a-47ef-99f4-cfe2679744b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:35.723048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0478f8-8a4a-47ef-99f4-cfe2679744b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:35.723421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0478f8-8a4a-47ef-99f4-cfe2679744b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:35.725883 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0478f8-8a4a-47ef-99f4-cfe2679744b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:35.730954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0478f8-8a4a-47ef-99f4-cfe2679744b3 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:35.733257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0303d8e14c724505b43abe95a63dcf3c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:35.759225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-424e490b-6814-4b66-95cf-1bb3aeab6006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0303d8e14c724505b43abe95a63dcf3c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:35.864883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 830/1672] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:35 2022] DELETE /identity/v3/users/0303d8e14c724505b43abe95a63dcf3c => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:35.866756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd400-e703-4bbb-91e1-b973a70517f0 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:35.866933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd400-e703-4bbb-91e1-b973a70517f0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:35.867310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fd400-e703-4bbb-91e1-b973a70517f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:35.931187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 843/1673] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:35 2022] POST /identity/v3/projects => generated 365 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:35.944923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21499f-053f-4aa2-b127-32a081769647 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:35.945776 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21499f-053f-4aa2-b127-32a081769647 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:35.946064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21499f-053f-4aa2-b127-32a081769647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:35.974026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e39fd400-e703-4bbb-91e1-b973a70517f0 None None] MFA Rules not processed for user `58303c18a6314dbeb1994d7813671e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:35.976403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e39fd400-e703-4bbb-91e1-b973a70517f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:36.055710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1c21499f-053f-4aa2-b127-32a081769647 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:36.065646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1c21499f-053f-4aa2-b127-32a081769647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:36.142018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:36.177590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 831/1674] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:36.180732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fd400-e703-4bbb-91e1-b973a70517f0 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:36.191530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e39fd400-e703-4bbb-91e1-b973a70517f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:36.315284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 844/1675] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:36.316329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21499f-053f-4aa2-b127-32a081769647 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:36.323934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1c21499f-053f-4aa2-b127-32a081769647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:36.349875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21499f-053f-4aa2-b127-32a081769647 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:36.356024 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25f715-ea9e-4e28-8f52-aa5092d8fec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:36.356922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25f715-ea9e-4e28-8f52-aa5092d8fec5 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:36.357403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25f715-ea9e-4e28-8f52-aa5092d8fec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:36.357828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25f715-ea9e-4e28-8f52-aa5092d8fec5 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:36.358686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25f715-ea9e-4e28-8f52-aa5092d8fec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:36.362773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25f715-ea9e-4e28-8f52-aa5092d8fec5 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:36.375861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fd400-e703-4bbb-91e1-b973a70517f0 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:36.390526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:36.391212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:36.391502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:36.392098 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] PATH_INFO: `/v3/projects/609dc8f22f75429c9d5506ba852411dc` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:36.394241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=609dc8f22f75429c9d5506ba852411dc)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:36.395839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:36.473778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 845/1676] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:36 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:36.476022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96de695-d1a1-4c85-aa50-b9e3a9db4269 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:36.479221 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96de695-d1a1-4c85-aa50-b9e3a9db4269 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:36.480214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96de695-d1a1-4c85-aa50-b9e3a9db4269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:36.536166 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d96de695-d1a1-4c85-aa50-b9e3a9db4269 None None] MFA Rules not processed for user `58303c18a6314dbeb1994d7813671e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:36.537922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d96de695-d1a1-4c85-aa50-b9e3a9db4269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:36.552488 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 846/1677] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:36.553666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96de695-d1a1-4c85-aa50-b9e3a9db4269 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:36.568549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d96de695-d1a1-4c85-aa50-b9e3a9db4269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:36.630539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609dc8f22f75429c9d5506ba852411dc', 'request_id': 'req-69271ee0-678c-429e-9bd3-068a8846f612'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:36.647838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609dc8f22f75429c9d5506ba852411dc', 'request_id': 'req-69271ee0-678c-429e-9bd3-068a8846f612'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:36.660763 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609dc8f22f75429c9d5506ba852411dc', 'request_id': 'req-69271ee0-678c-429e-9bd3-068a8846f612'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:36.759962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] The token cache is being invalidate because project 609dc8f22f75429c9d5506ba852411dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:36.760414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:36.761888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:36.762579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96de695-d1a1-4c85-aa50-b9e3a9db4269 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:36.764555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 832/1678] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:36 2022] DELETE /identity/v3/projects/609dc8f22f75429c9d5506ba852411dc => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:36.766653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:36.771267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:36.902218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262bb96-e5ac-4293-b0b4-e80054c0b1b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:36.902862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262bb96-e5ac-4293-b0b4-e80054c0b1b5 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:36.903138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262bb96-e5ac-4293-b0b4-e80054c0b1b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:36.903416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262bb96-e5ac-4293-b0b4-e80054c0b1b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:36.908801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7262bb96-e5ac-4293-b0b4-e80054c0b1b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:36.935550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69271ee0-678c-429e-9bd3-068a8846f612 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:36.936446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262bb96-e5ac-4293-b0b4-e80054c0b1b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:36.939415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86c78a-8277-45bd-aa5b-bb777e27a129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:36.940191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c78a-8277-45bd-aa5b-bb777e27a129 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:36.940538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c78a-8277-45bd-aa5b-bb777e27a129 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:36.940672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262bb96-e5ac-4293-b0b4-e80054c0b1b5 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:36.940940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c78a-8277-45bd-aa5b-bb777e27a129 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:36.941703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86c78a-8277-45bd-aa5b-bb777e27a129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:36.943445 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86c78a-8277-45bd-aa5b-bb777e27a129 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:36.950793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:36.958912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 833/1679] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:36.958912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5e83-58ca-417c-9476-e5192466cfd2 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:36.958912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5e83-58ca-417c-9476-e5192466cfd2 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:36.958912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b5e83-58ca-417c-9476-e5192466cfd2 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:36.960558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 834/1680] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 8 18:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:51:36.961961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b5e83-58ca-417c-9476-e5192466cfd2 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:36.963978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-311b5e83-58ca-417c-9476-e5192466cfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:36.964127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 847/1681] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:36.967992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262bb96-e5ac-4293-b0b4-e80054c0b1b5 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:36.975943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7262bb96-e5ac-4293-b0b4-e80054c0b1b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:36.996731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262bb96-e5ac-4293-b0b4-e80054c0b1b5 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:37.005004 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:37.009767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:37.010259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:37.011649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] PATH_INFO: `/v3/users/b27e66cff2c845e49d78e6dab365a6ae` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:37.014222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b27e66cff2c845e49d78e6dab365a6ae)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:37.015610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: WARNING py.warnings [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 18:51:37.015610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: warnings.warn(msg) Sep 08 18:51:37.015610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: Sep 08 18:51:37.017332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:37.092042 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b5e83-58ca-417c-9476-e5192466cfd2 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:37.099493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8952ced-d859-4206-af79-72bb1efb624d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:37.107747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8952ced-d859-4206-af79-72bb1efb624d service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:37.107747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8952ced-d859-4206-af79-72bb1efb624d service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:37.107747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8952ced-d859-4206-af79-72bb1efb624d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:37.107747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d8952ced-d859-4206-af79-72bb1efb624d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:37.194734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8952ced-d859-4206-af79-72bb1efb624d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:37.197358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8952ced-d859-4206-af79-72bb1efb624d service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:37.208717 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 835/1682] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:37.216875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8952ced-d859-4206-af79-72bb1efb624d service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:37.222712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8952ced-d859-4206-af79-72bb1efb624d service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:37.229975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff613c-6ab9-450d-a71e-02f8c8c9a66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:37.230263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff613c-6ab9-450d-a71e-02f8c8c9a66f admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:37.230844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff613c-6ab9-450d-a71e-02f8c8c9a66f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:37.231048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff613c-6ab9-450d-a71e-02f8c8c9a66f admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:37.231969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff613c-6ab9-450d-a71e-02f8c8c9a66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:37.233743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff613c-6ab9-450d-a71e-02f8c8c9a66f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:37.263227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 836/1683] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:37.265010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4adbc-dcd0-48dd-82b8-718fbb81474e None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:37.265677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4adbc-dcd0-48dd-82b8-718fbb81474e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:37.266100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4adbc-dcd0-48dd-82b8-718fbb81474e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:37.317994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c7d4adbc-dcd0-48dd-82b8-718fbb81474e None None] MFA Rules not processed for user `bcf30ef695154d3a97d7e589aca21233`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:37.319667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c7d4adbc-dcd0-48dd-82b8-718fbb81474e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:37.332773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 837/1684] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:51:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:37.333247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fec854-a4fd-4d26-8a6c-dee115c4ff14 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:37.333536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fec854-a4fd-4d26-8a6c-dee115c4ff14 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:37.333814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fec854-a4fd-4d26-8a6c-dee115c4ff14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:37.367951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-d8fec854-a4fd-4d26-8a6c-dee115c4ff14 None None] MFA Rules not processed for user `2b57c46bc37940718735201095453f16`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:37.369650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d8fec854-a4fd-4d26-8a6c-dee115c4ff14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:37.392541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b27e66cff2c845e49d78e6dab365a6ae'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:37.393409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b27e66cff2c845e49d78e6dab365a6ae'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:37.395282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b27e66cff2c845e49d78e6dab365a6ae'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:37.429100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b27e66cff2c845e49d78e6dab365a6ae'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:37.442955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:37.502836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b27e66cff2c845e49d78e6dab365a6ae'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:37.505823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 838/1685] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:37.506342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fec854-a4fd-4d26-8a6c-dee115c4ff14 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:37.512536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d8fec854-a4fd-4d26-8a6c-dee115c4ff14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:37.535078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fec854-a4fd-4d26-8a6c-dee115c4ff14 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:37.553202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a0622-88d5-4e1a-abb8-79f11648a134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:37.553202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a0622-88d5-4e1a-abb8-79f11648a134 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:37.553202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a0622-88d5-4e1a-abb8-79f11648a134 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:37.553202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a0622-88d5-4e1a-abb8-79f11648a134 admin admin] PATH_INFO: `/v3/projects/d23ba35e24794be29cf60e1edad2edd6/users/460e34193f8444078b79bddfb0dc85bd/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:37.595680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a0622-88d5-4e1a-abb8-79f11648a134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23ba35e24794be29cf60e1edad2edd6, user_id=460e34193f8444078b79bddfb0dc85bd, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:37.606350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a0622-88d5-4e1a-abb8-79f11648a134 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:37.626049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 848/1686] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:36 2022] DELETE /identity/v3/users/b27e66cff2c845e49d78e6dab365a6ae => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:37.626992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:37.629182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:37.710393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 839/1687] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:37 2022] PUT /identity/v3/projects/d23ba35e24794be29cf60e1edad2edd6/users/460e34193f8444078b79bddfb0dc85bd/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:37.712766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb293a-7989-40ce-822a-d3dbda543123 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:37.713019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb293a-7989-40ce-822a-d3dbda543123 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:37.713331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb293a-7989-40ce-822a-d3dbda543123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:37.778459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-cccb293a-7989-40ce-822a-d3dbda543123 None None] MFA Rules not processed for user `460e34193f8444078b79bddfb0dc85bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:37.779272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cccb293a-7989-40ce-822a-d3dbda543123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:37.811083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac856b3f-18c6-4fe6-838a-5198b83dc080 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:37.900880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 840/1688] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:37.901360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e3774-87a0-418a-8e8a-db475b8bc9bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:37.901360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e3774-87a0-418a-8e8a-db475b8bc9bd service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:37.901512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e16f0e-4252-4111-9700-d51e9c6380a6 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:37.901581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e3774-87a0-418a-8e8a-db475b8bc9bd service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:37.901581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e3774-87a0-418a-8e8a-db475b8bc9bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:37.901725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e16f0e-4252-4111-9700-d51e9c6380a6 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:37.901725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e16f0e-4252-4111-9700-d51e9c6380a6 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:37.902418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-122e3774-87a0-418a-8e8a-db475b8bc9bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:37.904326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 841/1689] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 8 18:51:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:51:37.905389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e16f0e-4252-4111-9700-d51e9c6380a6 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:37.907071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-13e16f0e-4252-4111-9700-d51e9c6380a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:37.915302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e3774-87a0-418a-8e8a-db475b8bc9bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:37.918624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e3774-87a0-418a-8e8a-db475b8bc9bd service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:37.937707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 849/1690] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:37.941350 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e3774-87a0-418a-8e8a-db475b8bc9bd service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:38.035150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e16f0e-4252-4111-9700-d51e9c6380a6 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:38.042581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:38.043358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:38.043719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:38.044115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] PATH_INFO: `/v3/projects/b5cd9ee33ccb4212b761bb487384aad6` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:38.046213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5cd9ee33ccb4212b761bb487384aad6)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:38.049668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:38.113569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e3774-87a0-418a-8e8a-db475b8bc9bd service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:38.128143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06a469-6511-4403-a983-ea8783a5945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:38.128820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06a469-6511-4403-a983-ea8783a5945b admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:38.129292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06a469-6511-4403-a983-ea8783a5945b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:38.129576 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06a469-6511-4403-a983-ea8783a5945b admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:38.130226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06a469-6511-4403-a983-ea8783a5945b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:38.132203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06a469-6511-4403-a983-ea8783a5945b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:38.141588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:38.270070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 850/1691] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:37 2022] POST /identity/v3/projects => generated 363 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:38.277159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d517b9a-db17-413f-813b-f4b16def7590 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:38.277535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d517b9a-db17-413f-813b-f4b16def7590 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:38.278280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d517b9a-db17-413f-813b-f4b16def7590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:38.284550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5cd9ee33ccb4212b761bb487384aad6', 'request_id': 'req-5d18ac63-60b7-4902-990e-cf05a40e48ca'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:38.305667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5cd9ee33ccb4212b761bb487384aad6', 'request_id': 'req-5d18ac63-60b7-4902-990e-cf05a40e48ca'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:38.322652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5cd9ee33ccb4212b761bb487384aad6', 'request_id': 'req-5d18ac63-60b7-4902-990e-cf05a40e48ca'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:38.373933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-9d517b9a-db17-413f-813b-f4b16def7590 None None] MFA Rules not processed for user `bcf30ef695154d3a97d7e589aca21233`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:38.376007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9d517b9a-db17-413f-813b-f4b16def7590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:38.445164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] The token cache is being invalidate because project b5cd9ee33ccb4212b761bb487384aad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:38.446140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:38.447513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:38.457385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 842/1692] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:37 2022] DELETE /identity/v3/projects/b5cd9ee33ccb4212b761bb487384aad6 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:38.458315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:38.463138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:38.554600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 851/1693] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:51:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:38.557736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5105403-e295-4efa-a97a-d06e394481d7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:38.558204 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5105403-e295-4efa-a97a-d06e394481d7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:38.558627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5105403-e295-4efa-a97a-d06e394481d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:38.585299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c5105403-e295-4efa-a97a-d06e394481d7 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:38.585299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c5105403-e295-4efa-a97a-d06e394481d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:38.624678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18ac63-60b7-4902-990e-cf05a40e48ca admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:38.726415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 852/1694] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:38.745505 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5105403-e295-4efa-a97a-d06e394481d7 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:38.749003 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c5105403-e295-4efa-a97a-d06e394481d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:38.760031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881224e-a247-4192-b660-19f7acfbe724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:38.761381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881224e-a247-4192-b660-19f7acfbe724 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:38.761809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881224e-a247-4192-b660-19f7acfbe724 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:38.762955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881224e-a247-4192-b660-19f7acfbe724 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:38.767408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881224e-a247-4192-b660-19f7acfbe724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:38.772830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881224e-a247-4192-b660-19f7acfbe724 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:38.781075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5105403-e295-4efa-a97a-d06e394481d7 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:38.809232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a418b70-19f9-4299-827d-9851ce93cbee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:38.809232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a418b70-19f9-4299-827d-9851ce93cbee service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:38.809232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a418b70-19f9-4299-827d-9851ce93cbee service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:38.809232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a418b70-19f9-4299-827d-9851ce93cbee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:38.836144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a418b70-19f9-4299-827d-9851ce93cbee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:38.836144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a418b70-19f9-4299-827d-9851ce93cbee service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:38.878024 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 853/1695] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:38.879497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a418b70-19f9-4299-827d-9851ce93cbee service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:38.881602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4a418b70-19f9-4299-827d-9851ce93cbee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:38.893823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 843/1696] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:38 2022] POST /identity/v3/users => generated 288 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:38.893823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881224e-a247-4192-b660-19f7acfbe724 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:38.897358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4881224e-a247-4192-b660-19f7acfbe724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:38.935083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a418b70-19f9-4299-827d-9851ce93cbee service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:38.944602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:38.946064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:38.946529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:38.946812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] PATH_INFO: `/v3/projects/27f129dbc0ca4a5da78ea1e317b5e718` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:38.948482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f129dbc0ca4a5da78ea1e317b5e718)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:38.950952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:39.069758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881224e-a247-4192-b660-19f7acfbe724 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:39.077151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676e254-4e03-4093-9ca7-a010fa4755a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:39.077923 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e254-4e03-4093-9ca7-a010fa4755a6 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:39.078412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e254-4e03-4093-9ca7-a010fa4755a6 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:39.078825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e254-4e03-4093-9ca7-a010fa4755a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:39.081608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c676e254-4e03-4093-9ca7-a010fa4755a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:39.114149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f129dbc0ca4a5da78ea1e317b5e718', 'request_id': 'req-4450d56c-d21e-45a3-b1ac-1f894c83ea37'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:39.137766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f129dbc0ca4a5da78ea1e317b5e718', 'request_id': 'req-4450d56c-d21e-45a3-b1ac-1f894c83ea37'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:39.198036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676e254-4e03-4093-9ca7-a010fa4755a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:39.199748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676e254-4e03-4093-9ca7-a010fa4755a6 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:39.225326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 844/1697] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:39.227957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de72b29-e8be-455f-9fc0-74f0130d95ac None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:39.228254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de72b29-e8be-455f-9fc0-74f0130d95ac None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:39.228535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de72b29-e8be-455f-9fc0-74f0130d95ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:39.232114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f129dbc0ca4a5da78ea1e317b5e718', 'request_id': 'req-4450d56c-d21e-45a3-b1ac-1f894c83ea37'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:39.256806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] The token cache is being invalidate because project 27f129dbc0ca4a5da78ea1e317b5e718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:39.259311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:39.260923 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:39.261455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:39.270971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 854/1698] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:38 2022] DELETE /identity/v3/projects/27f129dbc0ca4a5da78ea1e317b5e718 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:39.270971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:39.271161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-1de72b29-e8be-455f-9fc0-74f0130d95ac None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:39.271161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1de72b29-e8be-455f-9fc0-74f0130d95ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:39.271938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:39.474069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 845/1699] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:39.478932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450d56c-d21e-45a3-b1ac-1f894c83ea37 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:39.480630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de72b29-e8be-455f-9fc0-74f0130d95ac None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:39.485739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1de72b29-e8be-455f-9fc0-74f0130d95ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:39.485856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005b468-6919-43c4-af0d-4abf21190478 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:39.486796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005b468-6919-43c4-af0d-4abf21190478 service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:39.487312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005b468-6919-43c4-af0d-4abf21190478 service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:39.487819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005b468-6919-43c4-af0d-4abf21190478 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:39.491250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5005b468-6919-43c4-af0d-4abf21190478 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:39.510681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005b468-6919-43c4-af0d-4abf21190478 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:39.514111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005b468-6919-43c4-af0d-4abf21190478 service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:39.540561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de72b29-e8be-455f-9fc0-74f0130d95ac None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:39.551492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 855/1700] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:39.556645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005b468-6919-43c4-af0d-4abf21190478 service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:39.558256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5005b468-6919-43c4-af0d-4abf21190478 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:39.561163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c580cb-5d3e-4ae9-942b-ed98acf9a53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:39.563789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c580cb-5d3e-4ae9-942b-ed98acf9a53b admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:39.565255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c580cb-5d3e-4ae9-942b-ed98acf9a53b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:39.565255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c580cb-5d3e-4ae9-942b-ed98acf9a53b admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:39.568843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c580cb-5d3e-4ae9-942b-ed98acf9a53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:39.575569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c580cb-5d3e-4ae9-942b-ed98acf9a53b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:39.577419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005b468-6919-43c4-af0d-4abf21190478 service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:39.582535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:39.583276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:39.583566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:39.584038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] PATH_INFO: `/v3/users/fc27de50f6fa494986997a5efdf62af3` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:39.585551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc27de50f6fa494986997a5efdf62af3)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:39.599094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 846/1701] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:39.608500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:39.612077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6d3b8-9aeb-419d-bca6-408ac37f3d7c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:39.613417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6d3b8-9aeb-419d-bca6-408ac37f3d7c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:39.614327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6d3b8-9aeb-419d-bca6-408ac37f3d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:39.712386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-15a6d3b8-9aeb-419d-bca6-408ac37f3d7c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:39.717823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-15a6d3b8-9aeb-419d-bca6-408ac37f3d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:39.752578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 847/1702] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:39.753605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6d3b8-9aeb-419d-bca6-408ac37f3d7c None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:39.792714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6d3b8-9aeb-419d-bca6-408ac37f3d7c None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:39.808467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b1cd2-4fef-4add-b07f-b94835ea31c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:39.809289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b1cd2-4fef-4add-b07f-b94835ea31c3 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:39.810521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b1cd2-4fef-4add-b07f-b94835ea31c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:39.813371 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b1cd2-4fef-4add-b07f-b94835ea31c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:39.814223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b1cd2-4fef-4add-b07f-b94835ea31c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:39.816540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b1cd2-4fef-4add-b07f-b94835ea31c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:39.819704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b1cd2-4fef-4add-b07f-b94835ea31c3 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:39.858969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 848/1703] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:39.861155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b1cd2-4fef-4add-b07f-b94835ea31c3 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:39.877194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b1cd2-4fef-4add-b07f-b94835ea31c3 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:39.886685 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04562ff9-00ee-4ce4-9611-c042e3556945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:39.887912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04562ff9-00ee-4ce4-9611-c042e3556945 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:39.888345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04562ff9-00ee-4ce4-9611-c042e3556945 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:39.888765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04562ff9-00ee-4ce4-9611-c042e3556945 admin admin] PATH_INFO: `/v3/projects/2ee18227c3a44a4c8fd009a35bdd2aad/users/ddee36d0ce64498c8481d8d3c7364c42/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:39.924951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc27de50f6fa494986997a5efdf62af3'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:39.925857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04562ff9-00ee-4ce4-9611-c042e3556945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee18227c3a44a4c8fd009a35bdd2aad, user_id=ddee36d0ce64498c8481d8d3c7364c42, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:39.931741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc27de50f6fa494986997a5efdf62af3'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:39.936346 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04562ff9-00ee-4ce4-9611-c042e3556945 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:39.949255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc27de50f6fa494986997a5efdf62af3'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:39.977813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc27de50f6fa494986997a5efdf62af3'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:40.026355 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-cf1ee1b3-290b-4fd7-847c-ca4b934c09cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc27de50f6fa494986997a5efdf62af3'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:40.038527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 849/1704] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:39 2022] PUT /identity/v3/projects/2ee18227c3a44a4c8fd009a35bdd2aad/users/ddee36d0ce64498c8481d8d3c7364c42/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:40.039906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186327e6-414b-4e25-8942-6e9b228ea4e3 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:40.040670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186327e6-414b-4e25-8942-6e9b228ea4e3 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:40.041845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186327e6-414b-4e25-8942-6e9b228ea4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:40.074928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-186327e6-414b-4e25-8942-6e9b228ea4e3 None None] MFA Rules not processed for user `91e852d04f9f464ab8afbaeb9ff355e6`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:40.076394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-186327e6-414b-4e25-8942-6e9b228ea4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:40.109528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:40.137628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 856/1705] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:39 2022] DELETE /identity/v3/users/fc27de50f6fa494986997a5efdf62af3 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:40.137628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdc212-49d5-4a54-8c1c-8faad5f82e9c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:40.137628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdc212-49d5-4a54-8c1c-8faad5f82e9c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:40.137628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdc212-49d5-4a54-8c1c-8faad5f82e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:40.171320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-82cdc212-49d5-4a54-8c1c-8faad5f82e9c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:40.174158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-82cdc212-49d5-4a54-8c1c-8faad5f82e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:40.187742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 850/1706] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:40.190275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186327e6-414b-4e25-8942-6e9b228ea4e3 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:40.197479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-186327e6-414b-4e25-8942-6e9b228ea4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:40.258737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 857/1707] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:40.259807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdc212-49d5-4a54-8c1c-8faad5f82e9c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:40.271385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-82cdc212-49d5-4a54-8c1c-8faad5f82e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:40.302593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdc212-49d5-4a54-8c1c-8faad5f82e9c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:40.313965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b62e4-51d8-40a8-829f-cae1cb26f4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:40.314595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b62e4-51d8-40a8-829f-cae1cb26f4fa admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:40.314875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b62e4-51d8-40a8-829f-cae1cb26f4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:40.315147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b62e4-51d8-40a8-829f-cae1cb26f4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:40.316767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b62e4-51d8-40a8-829f-cae1cb26f4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:40.318924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b62e4-51d8-40a8-829f-cae1cb26f4fa admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:40.328134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186327e6-414b-4e25-8942-6e9b228ea4e3 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:40.333000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 858/1708] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:40.337716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891e69-d80e-4d9c-8bda-8a2c8da8af3b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:40.338003 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891e69-d80e-4d9c-8bda-8a2c8da8af3b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:40.338313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891e69-d80e-4d9c-8bda-8a2c8da8af3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:40.343756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0893b9-28f2-4d95-ae98-a7e647c3bfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:40.344391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0893b9-28f2-4d95-ae98-a7e647c3bfcb admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:40.344674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0893b9-28f2-4d95-ae98-a7e647c3bfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:40.344948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0893b9-28f2-4d95-ae98-a7e647c3bfcb admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:40.345616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0893b9-28f2-4d95-ae98-a7e647c3bfcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:40.347791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0893b9-28f2-4d95-ae98-a7e647c3bfcb admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:40.362759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-11891e69-d80e-4d9c-8bda-8a2c8da8af3b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:40.364580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-11891e69-d80e-4d9c-8bda-8a2c8da8af3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:40.382585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 859/1709] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:40.384489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11891e69-d80e-4d9c-8bda-8a2c8da8af3b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:40.426333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11891e69-d80e-4d9c-8bda-8a2c8da8af3b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:40.426333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fc0e2-f021-43df-9c4c-a24b6271cd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:40.426333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fc0e2-f021-43df-9c4c-a24b6271cd3c admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:40.426333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fc0e2-f021-43df-9c4c-a24b6271cd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:40.427672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fc0e2-f021-43df-9c4c-a24b6271cd3c admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:40.427672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fc0e2-f021-43df-9c4c-a24b6271cd3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:40.428064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fc0e2-f021-43df-9c4c-a24b6271cd3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:40.433523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fc0e2-f021-43df-9c4c-a24b6271cd3c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:40.465564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 860/1710] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:40.466765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fc0e2-f021-43df-9c4c-a24b6271cd3c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:40.489503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fc0e2-f021-43df-9c4c-a24b6271cd3c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:40.501787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90604f23-afd5-488c-8b16-b73b5d920aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:40.502725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90604f23-afd5-488c-8b16-b73b5d920aae admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:40.502725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90604f23-afd5-488c-8b16-b73b5d920aae admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:40.503013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90604f23-afd5-488c-8b16-b73b5d920aae admin admin] PATH_INFO: `/v3/projects/2ee18227c3a44a4c8fd009a35bdd2aad/users/ddee36d0ce64498c8481d8d3c7364c42/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:40.510695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90604f23-afd5-488c-8b16-b73b5d920aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee18227c3a44a4c8fd009a35bdd2aad, user_id=ddee36d0ce64498c8481d8d3c7364c42, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:40.517970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90604f23-afd5-488c-8b16-b73b5d920aae admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:40.538149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 861/1711] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:40 2022] PUT /identity/v3/projects/2ee18227c3a44a4c8fd009a35bdd2aad/users/ddee36d0ce64498c8481d8d3c7364c42/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:40.542962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90604f23-afd5-488c-8b16-b73b5d920aae admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:40.547290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-90604f23-afd5-488c-8b16-b73b5d920aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:40.592093 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 851/1712] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:40 2022] POST /identity/v3/projects => generated 373 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:40.595749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0893b9-28f2-4d95-ae98-a7e647c3bfcb admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:40.746905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90604f23-afd5-488c-8b16-b73b5d920aae admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:40.863600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0893b9-28f2-4d95-ae98-a7e647c3bfcb admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:40.875902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc77e0-d9cd-4401-9584-a28cc8812e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:40.876822 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc77e0-d9cd-4401-9584-a28cc8812e31 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:40.877393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc77e0-d9cd-4401-9584-a28cc8812e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:40.877869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc77e0-d9cd-4401-9584-a28cc8812e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:40.881416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23841126-e702-4da7-b7b2-77836a593e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:40.889240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23841126-e702-4da7-b7b2-77836a593e07 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:40.889240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23841126-e702-4da7-b7b2-77836a593e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:40.889240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23841126-e702-4da7-b7b2-77836a593e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:40.889240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23841126-e702-4da7-b7b2-77836a593e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:40.889240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23841126-e702-4da7-b7b2-77836a593e07 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:40.905450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc77e0-d9cd-4401-9584-a28cc8812e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:40.906928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc77e0-d9cd-4401-9584-a28cc8812e31 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:40.934906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 862/1713] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:40.936432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b4bb0-b7f4-4f47-b457-18c8798ab0dc None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:40.936715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b4bb0-b7f4-4f47-b457-18c8798ab0dc None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:40.936997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b4bb0-b7f4-4f47-b457-18c8798ab0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.006367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8a2b4bb0-b7f4-4f47-b457-18c8798ab0dc None None] MFA Rules not processed for user `ddee36d0ce64498c8481d8d3c7364c42`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:41.006907 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 852/1714] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:40 2022] POST /identity/v3/projects => generated 375 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:41.010869 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23841126-e702-4da7-b7b2-77836a593e07 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:41.020130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8a2b4bb0-b7f4-4f47-b457-18c8798ab0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:41.166716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:41.231964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23841126-e702-4da7-b7b2-77836a593e07 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:41.245705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295a049-4858-4964-a43f-d6dbe1cecdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:41.246676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295a049-4858-4964-a43f-d6dbe1cecdc9 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:41.246676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295a049-4858-4964-a43f-d6dbe1cecdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:41.248804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295a049-4858-4964-a43f-d6dbe1cecdc9 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.249534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295a049-4858-4964-a43f-d6dbe1cecdc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:41.252827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295a049-4858-4964-a43f-d6dbe1cecdc9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:41.295132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 863/1715] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:41.297105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b4bb0-b7f4-4f47-b457-18c8798ab0dc None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:41.332602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b4bb0-b7f4-4f47-b457-18c8798ab0dc None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:41.341302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8427f-16cf-4508-a39f-4f0500b62b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:41.342308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8427f-16cf-4508-a39f-4f0500b62b07 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:41.342779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8427f-16cf-4508-a39f-4f0500b62b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:41.343495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8427f-16cf-4508-a39f-4f0500b62b07 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.344432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8427f-16cf-4508-a39f-4f0500b62b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:41.351374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8427f-16cf-4508-a39f-4f0500b62b07 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:41.489002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 853/1716] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:41 2022] POST /identity/v3/users => generated 287 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:41.492990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295a049-4858-4964-a43f-d6dbe1cecdc9 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:41.499497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a295a049-4858-4964-a43f-d6dbe1cecdc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:41.545429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 864/1717] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:41 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:41.550276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8427f-16cf-4508-a39f-4f0500b62b07 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:41.573972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8427f-16cf-4508-a39f-4f0500b62b07 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:41.582743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1cdcc-d578-44ee-89db-51294ada2f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:41.583365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1cdcc-d578-44ee-89db-51294ada2f2e admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:41.583651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1cdcc-d578-44ee-89db-51294ada2f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:41.583994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1cdcc-d578-44ee-89db-51294ada2f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.584642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1cdcc-d578-44ee-89db-51294ada2f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:41.587001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1cdcc-d578-44ee-89db-51294ada2f2e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:41.601452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 865/1718] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:41.602850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1cdcc-d578-44ee-89db-51294ada2f2e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:41.619232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1cdcc-d578-44ee-89db-51294ada2f2e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:41.629975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4571cd-2fc5-4b16-9701-67b2294293f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:41.630731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4571cd-2fc5-4b16-9701-67b2294293f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:41.630933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4571cd-2fc5-4b16-9701-67b2294293f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:41.631178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4571cd-2fc5-4b16-9701-67b2294293f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.631859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4571cd-2fc5-4b16-9701-67b2294293f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:41.634734 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4571cd-2fc5-4b16-9701-67b2294293f4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:41.635863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295a049-4858-4964-a43f-d6dbe1cecdc9 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:41.645236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da23db-6dbf-4ae7-900c-8a78d36c520d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:41.648158 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da23db-6dbf-4ae7-900c-8a78d36c520d service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:41.648823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da23db-6dbf-4ae7-900c-8a78d36c520d service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:41.649534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da23db-6dbf-4ae7-900c-8a78d36c520d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.649903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 866/1719] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:41.650906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4571cd-2fc5-4b16-9701-67b2294293f4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:41.663024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da23db-6dbf-4ae7-900c-8a78d36c520d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:41.665551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da23db-6dbf-4ae7-900c-8a78d36c520d service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:41.666896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4571cd-2fc5-4b16-9701-67b2294293f4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:41.673934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd835f6a-25fa-4e87-a6c4-fd525c920be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:41.674791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd835f6a-25fa-4e87-a6c4-fd525c920be9 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:41.675177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd835f6a-25fa-4e87-a6c4-fd525c920be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:41.675554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd835f6a-25fa-4e87-a6c4-fd525c920be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.676674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd835f6a-25fa-4e87-a6c4-fd525c920be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:41.692058 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd835f6a-25fa-4e87-a6c4-fd525c920be9 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:41.696998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 854/1720] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:41.697916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da23db-6dbf-4ae7-900c-8a78d36c520d service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:41.724687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 867/1721] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:41.725887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd835f6a-25fa-4e87-a6c4-fd525c920be9 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:41.740702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da23db-6dbf-4ae7-900c-8a78d36c520d service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:41.755304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd835f6a-25fa-4e87-a6c4-fd525c920be9 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:41.765802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32250f06-ead9-46a5-ac1b-cf0df4e89ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:41.772013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32250f06-ead9-46a5-ac1b-cf0df4e89ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:41.772013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32250f06-ead9-46a5-ac1b-cf0df4e89ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:41.772013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32250f06-ead9-46a5-ac1b-cf0df4e89ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.772013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32250f06-ead9-46a5-ac1b-cf0df4e89ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:41.772654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fa419-fc50-4568-89f7-446523ceb73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:41.772654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fa419-fc50-4568-89f7-446523ceb73f admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:41.772654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fa419-fc50-4568-89f7-446523ceb73f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:41.772654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fa419-fc50-4568-89f7-446523ceb73f admin admin] PATH_INFO: `/v3/projects/fa60d4abcb9443c09325824819dcec1a/users/05a00bd2c7614d50bf2d9c27f34a479f/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.781473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32250f06-ead9-46a5-ac1b-cf0df4e89ec9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:41.814386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fa419-fc50-4568-89f7-446523ceb73f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa60d4abcb9443c09325824819dcec1a, user_id=05a00bd2c7614d50bf2d9c27f34a479f, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:41.824269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fa419-fc50-4568-89f7-446523ceb73f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:41.828986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 855/1722] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:41.836725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32250f06-ead9-46a5-ac1b-cf0df4e89ec9 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:41.874327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32250f06-ead9-46a5-ac1b-cf0df4e89ec9 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:41.901867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3fd34-767b-49f2-835e-862a364d3bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:41.902579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3fd34-767b-49f2-835e-862a364d3bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:41.902864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3fd34-767b-49f2-835e-862a364d3bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:41.903141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3fd34-767b-49f2-835e-862a364d3bc8 admin admin] PATH_INFO: `/v3/projects/f6cfe6743951447daa85f166c8cd5644/users/f0964faec07c487da00b184294d818d3/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.917916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 868/1723] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:41 2022] PUT /identity/v3/projects/fa60d4abcb9443c09325824819dcec1a/users/05a00bd2c7614d50bf2d9c27f34a479f/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:41.932150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba603f-1692-4716-8bfd-7b01f794b0e5 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:41.932695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba603f-1692-4716-8bfd-7b01f794b0e5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:41.933213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba603f-1692-4716-8bfd-7b01f794b0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:41.968481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3fd34-767b-49f2-835e-862a364d3bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6cfe6743951447daa85f166c8cd5644, user_id=f0964faec07c487da00b184294d818d3, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:41.975095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3fd34-767b-49f2-835e-862a364d3bc8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:41.998965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d2ba603f-1692-4716-8bfd-7b01f794b0e5 None None] MFA Rules not processed for user `05a00bd2c7614d50bf2d9c27f34a479f`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:42.001006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d2ba603f-1692-4716-8bfd-7b01f794b0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:42.061506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 856/1724] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:41 2022] PUT /identity/v3/projects/f6cfe6743951447daa85f166c8cd5644/users/f0964faec07c487da00b184294d818d3/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:42.074610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1fe06-64ba-4112-8198-342a00e47758 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:42.074903 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1fe06-64ba-4112-8198-342a00e47758 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:42.075178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1fe06-64ba-4112-8198-342a00e47758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:42.146525 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a3d1fe06-64ba-4112-8198-342a00e47758 None None] MFA Rules not processed for user `f0964faec07c487da00b184294d818d3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:42.157074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a3d1fe06-64ba-4112-8198-342a00e47758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:42.222702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:42.232071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 869/1725] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:42.237499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba603f-1692-4716-8bfd-7b01f794b0e5 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:42.238204 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d2ba603f-1692-4716-8bfd-7b01f794b0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:42.399097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba603f-1692-4716-8bfd-7b01f794b0e5 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:42.419467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 857/1726] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:51:42.423892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1fe06-64ba-4112-8198-342a00e47758 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:42.532567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d8c14-15bd-4382-9a5d-c263f354f324 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:42.533243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d8c14-15bd-4382-9a5d-c263f354f324 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:42.533528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d8c14-15bd-4382-9a5d-c263f354f324 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:42.533804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d8c14-15bd-4382-9a5d-c263f354f324 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:42.537106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-bb0d8c14-15bd-4382-9a5d-c263f354f324 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:42.623018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1fe06-64ba-4112-8198-342a00e47758 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:42.629463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c7d72-fdc5-49b2-950e-a03622bfb54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:42.630101 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c7d72-fdc5-49b2-950e-a03622bfb54d admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:42.630385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c7d72-fdc5-49b2-950e-a03622bfb54d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:42.630821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c7d72-fdc5-49b2-950e-a03622bfb54d admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:42.631479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c7d72-fdc5-49b2-950e-a03622bfb54d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:42.634819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c7d72-fdc5-49b2-950e-a03622bfb54d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:42.643918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d8c14-15bd-4382-9a5d-c263f354f324 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:42.644134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:42.650108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d8c14-15bd-4382-9a5d-c263f354f324 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:42.670793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 870/1727] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:42.671994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d8c14-15bd-4382-9a5d-c263f354f324 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:42.677113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-bb0d8c14-15bd-4382-9a5d-c263f354f324 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:42.707100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d8c14-15bd-4382-9a5d-c263f354f324 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:42.722543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:42.723363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:42.723648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:42.723933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] PATH_INFO: `/v3/projects/ebca8e24f4c846bfb316d57be7526fd1` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:42.725282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebca8e24f4c846bfb316d57be7526fd1)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:42.728063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:42.749731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 858/1728] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:42 2022] POST /identity/v3/projects => generated 373 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:42.751150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c7d72-fdc5-49b2-950e-a03622bfb54d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:42.837809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c7d72-fdc5-49b2-950e-a03622bfb54d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:42.842767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedbd50-d703-41a9-b377-7e7b070975f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:42.843476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedbd50-d703-41a9-b377-7e7b070975f8 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:42.843828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedbd50-d703-41a9-b377-7e7b070975f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:42.844173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedbd50-d703-41a9-b377-7e7b070975f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:42.844914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedbd50-d703-41a9-b377-7e7b070975f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:42.846980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedbd50-d703-41a9-b377-7e7b070975f8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:43.035188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 859/1729] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:42 2022] POST /identity/v3/projects => generated 375 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:43.036278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedbd50-d703-41a9-b377-7e7b070975f8 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:43.063014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebca8e24f4c846bfb316d57be7526fd1', 'request_id': 'req-263782ab-19eb-405d-ade1-fa8f465181e9'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:43.080444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebca8e24f4c846bfb316d57be7526fd1', 'request_id': 'req-263782ab-19eb-405d-ade1-fa8f465181e9'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:43.173457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebca8e24f4c846bfb316d57be7526fd1', 'request_id': 'req-263782ab-19eb-405d-ade1-fa8f465181e9'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:43.189156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedbd50-d703-41a9-b377-7e7b070975f8 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:43.191494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] The token cache is being invalidate because project ebca8e24f4c846bfb316d57be7526fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:43.191494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:43.192620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:43.194671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 871/1730] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:42 2022] DELETE /identity/v3/projects/ebca8e24f4c846bfb316d57be7526fd1 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:43.195856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:43.198666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:43.203760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3092f36-598a-4af8-88ac-4cbd17ccae16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:43.204389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3092f36-598a-4af8-88ac-4cbd17ccae16 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:43.204673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3092f36-598a-4af8-88ac-4cbd17ccae16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:43.204954 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3092f36-598a-4af8-88ac-4cbd17ccae16 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:43.205650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3092f36-598a-4af8-88ac-4cbd17ccae16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:43.215633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3092f36-598a-4af8-88ac-4cbd17ccae16 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:43.359534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263782ab-19eb-405d-ade1-fa8f465181e9 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:43.366458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8710e5-8ad8-421f-9df4-976cc85b5b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:43.368640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8710e5-8ad8-421f-9df4-976cc85b5b6d admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:43.369140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8710e5-8ad8-421f-9df4-976cc85b5b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:43.369919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8710e5-8ad8-421f-9df4-976cc85b5b6d admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:43.370390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8710e5-8ad8-421f-9df4-976cc85b5b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:43.374168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8710e5-8ad8-421f-9df4-976cc85b5b6d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:43.378868 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 860/1731] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:43 2022] POST /identity/v3/users => generated 293 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:43.388729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a3c97-fe15-4b6b-a98d-2e034fe53b5c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:43.388729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a3c97-fe15-4b6b-a98d-2e034fe53b5c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:43.388729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a3c97-fe15-4b6b-a98d-2e034fe53b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:43.439378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-9a9a3c97-fe15-4b6b-a98d-2e034fe53b5c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:43.441162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9a9a3c97-fe15-4b6b-a98d-2e034fe53b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:43.456980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 861/1732] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:43.457996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a3c97-fe15-4b6b-a98d-2e034fe53b5c None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:43.459745 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9a9a3c97-fe15-4b6b-a98d-2e034fe53b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:43.475717 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a3c97-fe15-4b6b-a98d-2e034fe53b5c None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:43.478716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195eb3f-1b32-4eb8-9b1d-bee09875f168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:43.479412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195eb3f-1b32-4eb8-9b1d-bee09875f168 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:43.479769 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195eb3f-1b32-4eb8-9b1d-bee09875f168 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:43.480115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195eb3f-1b32-4eb8-9b1d-bee09875f168 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:43.480864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195eb3f-1b32-4eb8-9b1d-bee09875f168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:43.482745 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195eb3f-1b32-4eb8-9b1d-bee09875f168 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:43.498288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 862/1733] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:43.498288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35c567-6fc4-4181-93d3-9d6533f319a1 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:43.498288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35c567-6fc4-4181-93d3-9d6533f319a1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:43.498288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35c567-6fc4-4181-93d3-9d6533f319a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:43.538118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 872/1734] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:43 2022] POST /identity/v3/users => generated 294 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:43.539380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8710e5-8ad8-421f-9df4-976cc85b5b6d admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:43.549234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-0e35c567-6fc4-4181-93d3-9d6533f319a1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:43.550685 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0e35c567-6fc4-4181-93d3-9d6533f319a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:43.562676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8710e5-8ad8-421f-9df4-976cc85b5b6d admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:43.571667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3cce5-c420-472c-a121-60e8dbdcaa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:43.575083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 863/1735] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:43.575782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3cce5-c420-472c-a121-60e8dbdcaa79 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:43.575934 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35c567-6fc4-4181-93d3-9d6533f319a1 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:43.576372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3cce5-c420-472c-a121-60e8dbdcaa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:43.576916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3cce5-c420-472c-a121-60e8dbdcaa79 admin admin] PATH_INFO: `/v3/projects/033ff712076744e7b1313eee01d23692/users/fb979fdaa7c442458183ec52a642a248/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:43.637737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3cce5-c420-472c-a121-60e8dbdcaa79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033ff712076744e7b1313eee01d23692, user_id=fb979fdaa7c442458183ec52a642a248, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:43.645697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3cce5-c420-472c-a121-60e8dbdcaa79 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:43.659605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35c567-6fc4-4181-93d3-9d6533f319a1 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:43.697771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1963b-78b6-4f86-af81-f37288fdf126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:43.698523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1963b-78b6-4f86-af81-f37288fdf126 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:43.698911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1963b-78b6-4f86-af81-f37288fdf126 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:43.699293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1963b-78b6-4f86-af81-f37288fdf126 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:43.700239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1963b-78b6-4f86-af81-f37288fdf126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:43.704222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1963b-78b6-4f86-af81-f37288fdf126 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:43.737106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 864/1736] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:43.737106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1963b-78b6-4f86-af81-f37288fdf126 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:43.761350 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 873/1737] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:43 2022] PUT /identity/v3/projects/033ff712076744e7b1313eee01d23692/users/fb979fdaa7c442458183ec52a642a248/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:43.762473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3cce5-c420-472c-a121-60e8dbdcaa79 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:43.770703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1963b-78b6-4f86-af81-f37288fdf126 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:43.931700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38c47f-07fd-4ca8-8fcf-fdafbe38df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:43.932524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38c47f-07fd-4ca8-8fcf-fdafbe38df27 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:43.932867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38c47f-07fd-4ca8-8fcf-fdafbe38df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:43.933311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38c47f-07fd-4ca8-8fcf-fdafbe38df27 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:43.934166 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38c47f-07fd-4ca8-8fcf-fdafbe38df27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:43.934399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38c47f-07fd-4ca8-8fcf-fdafbe38df27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:43.939077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3cce5-c420-472c-a121-60e8dbdcaa79 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:43.942530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38c47f-07fd-4ca8-8fcf-fdafbe38df27 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:43.956156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54176604-02a7-4664-b57d-1406df6914c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:43.958516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54176604-02a7-4664-b57d-1406df6914c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:43.958955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54176604-02a7-4664-b57d-1406df6914c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:43.959243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54176604-02a7-4664-b57d-1406df6914c4 admin admin] PATH_INFO: `/v3/projects/01b9121bdb0848d9b764e026941ce67a/users/bb5ac9d0ce97480bbf0d7d85e9bae65f/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:43.978358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 865/1738] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:43.978358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38c47f-07fd-4ca8-8fcf-fdafbe38df27 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:43.994311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38c47f-07fd-4ca8-8fcf-fdafbe38df27 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:44.003063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0e05-9eed-40be-8828-42e8688b7b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:44.007069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0e05-9eed-40be-8828-42e8688b7b1b admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:44.007069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0e05-9eed-40be-8828-42e8688b7b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:44.007069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0e05-9eed-40be-8828-42e8688b7b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:44.007069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e0e05-9eed-40be-8828-42e8688b7b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:44.010343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e0e05-9eed-40be-8828-42e8688b7b1b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:44.023641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 866/1739] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:44.025911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54176604-02a7-4664-b57d-1406df6914c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b9121bdb0848d9b764e026941ce67a, user_id=bb5ac9d0ce97480bbf0d7d85e9bae65f, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:44.026641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0e05-9eed-40be-8828-42e8688b7b1b admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:44.034991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54176604-02a7-4664-b57d-1406df6914c4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:44.043382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0e05-9eed-40be-8828-42e8688b7b1b admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:44.049443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4429f-4a96-42c3-b2b3-62a8b18be4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:44.050644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4429f-4a96-42c3-b2b3-62a8b18be4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:44.051274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4429f-4a96-42c3-b2b3-62a8b18be4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:44.057129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4429f-4a96-42c3-b2b3-62a8b18be4a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:44.057129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4429f-4a96-42c3-b2b3-62a8b18be4a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:44.057129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4429f-4a96-42c3-b2b3-62a8b18be4a7 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:44.167723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 874/1740] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:43 2022] PUT /identity/v3/projects/01b9121bdb0848d9b764e026941ce67a/users/bb5ac9d0ce97480bbf0d7d85e9bae65f/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:44.169651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54176604-02a7-4664-b57d-1406df6914c4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:44.175562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 867/1741] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:44 2022] POST /identity/v3/projects => generated 405 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:44.177299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4429f-4a96-42c3-b2b3-62a8b18be4a7 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:44.326063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54176604-02a7-4664-b57d-1406df6914c4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:44.332930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51763fa-7adb-4429-99c6-e58510092044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:44.337730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51763fa-7adb-4429-99c6-e58510092044 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:44.339267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51763fa-7adb-4429-99c6-e58510092044 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:44.339508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4429f-4a96-42c3-b2b3-62a8b18be4a7 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:44.340038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51763fa-7adb-4429-99c6-e58510092044 admin admin] PATH_INFO: `/v3/projects/033ff712076744e7b1313eee01d23692/users/fb979fdaa7c442458183ec52a642a248/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:44.344644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fdb72-57c9-407d-b891-826a4767de21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:44.347643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fdb72-57c9-407d-b891-826a4767de21 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:44.347643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fdb72-57c9-407d-b891-826a4767de21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:44.347643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fdb72-57c9-407d-b891-826a4767de21 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:44.347643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fdb72-57c9-407d-b891-826a4767de21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:44.347947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51763fa-7adb-4429-99c6-e58510092044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033ff712076744e7b1313eee01d23692, user_id=fb979fdaa7c442458183ec52a642a248, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:44.348420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fdb72-57c9-407d-b891-826a4767de21 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:44.351091 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51763fa-7adb-4429-99c6-e58510092044 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:44.386904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 868/1742] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:44.387988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fdb72-57c9-407d-b891-826a4767de21 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:44.396400 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 875/1743] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:44 2022] PUT /identity/v3/projects/033ff712076744e7b1313eee01d23692/users/fb979fdaa7c442458183ec52a642a248/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:44.397284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51763fa-7adb-4429-99c6-e58510092044 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:44.427124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fdb72-57c9-407d-b891-826a4767de21 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:44.603840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:44.610243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51763fa-7adb-4429-99c6-e58510092044 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:44.615250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b9045-083d-4a08-8c49-d3353e1d6f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:44.615423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d512662-6c05-40c7-8285-888db7fc800f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:44.616097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d512662-6c05-40c7-8285-888db7fc800f admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:44.616220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b9045-083d-4a08-8c49-d3353e1d6f85 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:44.616363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d512662-6c05-40c7-8285-888db7fc800f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:44.616646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d512662-6c05-40c7-8285-888db7fc800f admin admin] PATH_INFO: `/v3/projects/01b9121bdb0848d9b764e026941ce67a/users/bb5ac9d0ce97480bbf0d7d85e9bae65f/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:44.617590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b9045-083d-4a08-8c49-d3353e1d6f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:44.619447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d512662-6c05-40c7-8285-888db7fc800f admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b9121bdb0848d9b764e026941ce67a, user_id=bb5ac9d0ce97480bbf0d7d85e9bae65f, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:44.620011 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b9045-083d-4a08-8c49-d3353e1d6f85 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:44.620763 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b9045-083d-4a08-8c49-d3353e1d6f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:44.622961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b9045-083d-4a08-8c49-d3353e1d6f85 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:44.627149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d512662-6c05-40c7-8285-888db7fc800f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:44.650308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 876/1744] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:44 2022] PUT /identity/v3/projects/01b9121bdb0848d9b764e026941ce67a/users/bb5ac9d0ce97480bbf0d7d85e9bae65f/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:44.663550 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476be0d-bec3-4985-b238-09660f655f52 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:44.663790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476be0d-bec3-4985-b238-09660f655f52 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:44.665374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476be0d-bec3-4985-b238-09660f655f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:44.730689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7476be0d-bec3-4985-b238-09660f655f52 None None] MFA Rules not processed for user `fb979fdaa7c442458183ec52a642a248`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:44.733899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7476be0d-bec3-4985-b238-09660f655f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:44.794315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 869/1745] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:44 2022] POST /identity/v3/users => generated 303 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:44.795761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a034831-2aa9-48e0-8b77-2bfe50087181 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:44.796004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a034831-2aa9-48e0-8b77-2bfe50087181 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:44.796289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a034831-2aa9-48e0-8b77-2bfe50087181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:44.846133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2a034831-2aa9-48e0-8b77-2bfe50087181 None None] MFA Rules not processed for user `2b57c46bc37940718735201095453f16`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:44.847810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2a034831-2aa9-48e0-8b77-2bfe50087181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:44.876059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:44.898491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 877/1746] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:44.904001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d959fe1-2a13-4b89-9656-549f5df33a11 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:44.904626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d959fe1-2a13-4b89-9656-549f5df33a11 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:44.905109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d959fe1-2a13-4b89-9656-549f5df33a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:44.966284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-9d959fe1-2a13-4b89-9656-549f5df33a11 None None] MFA Rules not processed for user `bb5ac9d0ce97480bbf0d7d85e9bae65f`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:44.966284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9d959fe1-2a13-4b89-9656-549f5df33a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:44.994491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 870/1747] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:44.997873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a034831-2aa9-48e0-8b77-2bfe50087181 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:45.141397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 878/1748] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:45.145315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d959fe1-2a13-4b89-9656-549f5df33a11 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:45.152827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9d959fe1-2a13-4b89-9656-549f5df33a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:45.164452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a034831-2aa9-48e0-8b77-2bfe50087181 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:45.173285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15154753-8b5b-4799-b6bb-3672449fb2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:45.174109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15154753-8b5b-4799-b6bb-3672449fb2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:45.174792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15154753-8b5b-4799-b6bb-3672449fb2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:45.174902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15154753-8b5b-4799-b6bb-3672449fb2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:45.175490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15154753-8b5b-4799-b6bb-3672449fb2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:45.177262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15154753-8b5b-4799-b6bb-3672449fb2d4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:45.208206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 871/1749] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:45.213034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15154753-8b5b-4799-b6bb-3672449fb2d4 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:45.224706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15154753-8b5b-4799-b6bb-3672449fb2d4 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:45.233253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebb0fb-3801-4afb-ad64-99b8bda10341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:45.234644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebb0fb-3801-4afb-ad64-99b8bda10341 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:45.235375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebb0fb-3801-4afb-ad64-99b8bda10341 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:45.235861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebb0fb-3801-4afb-ad64-99b8bda10341 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:45.236779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebb0fb-3801-4afb-ad64-99b8bda10341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:45.240782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebb0fb-3801-4afb-ad64-99b8bda10341 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:45.253428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 872/1750] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:45.263238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d959fe1-2a13-4b89-9656-549f5df33a11 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:45.291014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebb0fb-3801-4afb-ad64-99b8bda10341 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:45.303762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca9805-a363-4847-b43a-f6bd304077d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:45.306969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca9805-a363-4847-b43a-f6bd304077d0 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:45.306969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca9805-a363-4847-b43a-f6bd304077d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:45.306969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca9805-a363-4847-b43a-f6bd304077d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:45.347438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebb0fb-3801-4afb-ad64-99b8bda10341 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:45.354067 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca9805-a363-4847-b43a-f6bd304077d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:45.355268 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca9805-a363-4847-b43a-f6bd304077d0 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:45.357031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283c5d8-2433-4170-891f-d1caa34b18e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:45.358661 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c5d8-2433-4170-891f-d1caa34b18e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:45.359426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c5d8-2433-4170-891f-d1caa34b18e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:45.360156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c5d8-2433-4170-891f-d1caa34b18e0 admin admin] PATH_INFO: `/v3/projects/6f85922076484272833f13ba1f95862b/users/e1bf6afb94664ec1abc8a544241ba7f3/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:45.368155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:45.372999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 879/1751] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:45.385601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283c5d8-2433-4170-891f-d1caa34b18e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f85922076484272833f13ba1f95862b, user_id=e1bf6afb94664ec1abc8a544241ba7f3, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:45.390253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283c5d8-2433-4170-891f-d1caa34b18e0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:45.533077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8137236-404e-4f82-9e99-2a8246780c4c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:45.533377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8137236-404e-4f82-9e99-2a8246780c4c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:45.533648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8137236-404e-4f82-9e99-2a8246780c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:45.539782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 873/1752] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:45 2022] PUT /identity/v3/projects/6f85922076484272833f13ba1f95862b/users/e1bf6afb94664ec1abc8a544241ba7f3/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:45.551314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b153d0-789b-4053-85df-b25718237ea9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:45.551598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b153d0-789b-4053-85df-b25718237ea9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:45.551874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b153d0-789b-4053-85df-b25718237ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:45.617366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-24b153d0-789b-4053-85df-b25718237ea9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:45.618998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-24b153d0-789b-4053-85df-b25718237ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:45.621768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:45.710386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f8137236-404e-4f82-9e99-2a8246780c4c None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:45.713086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f8137236-404e-4f82-9e99-2a8246780c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:45.791609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 874/1753] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:45.795474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b153d0-789b-4053-85df-b25718237ea9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:45.871027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 880/1754] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:51:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:45.871930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8137236-404e-4f82-9e99-2a8246780c4c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:45.877014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f8137236-404e-4f82-9e99-2a8246780c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:45.974767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b153d0-789b-4053-85df-b25718237ea9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:45.984141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854a7c5-acd6-4427-9fc5-fc0a3570dfb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:45.985036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854a7c5-acd6-4427-9fc5-fc0a3570dfb8 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:45.985337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854a7c5-acd6-4427-9fc5-fc0a3570dfb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:45.985778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854a7c5-acd6-4427-9fc5-fc0a3570dfb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:45.994025 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0854a7c5-acd6-4427-9fc5-fc0a3570dfb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:46.037376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8137236-404e-4f82-9e99-2a8246780c4c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:46.042879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba6d0b-d06c-4054-a166-dafbb05970d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:46.043699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba6d0b-d06c-4054-a166-dafbb05970d7 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:46.046502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba6d0b-d06c-4054-a166-dafbb05970d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:46.046802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba6d0b-d06c-4054-a166-dafbb05970d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:46.061140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba6d0b-d06c-4054-a166-dafbb05970d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:46.062926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba6d0b-d06c-4054-a166-dafbb05970d7 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:46.105175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 881/1755] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:46.121159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854a7c5-acd6-4427-9fc5-fc0a3570dfb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:46.123038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854a7c5-acd6-4427-9fc5-fc0a3570dfb8 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:46.135802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 875/1756] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:46.395671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba6d0b-d06c-4054-a166-dafbb05970d7 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:46.395671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e8ba6d0b-d06c-4054-a166-dafbb05970d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:46.542615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba6d0b-d06c-4054-a166-dafbb05970d7 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:46.548686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39507755-5ff4-48d9-99d0-716b69a0a190 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:46.549455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39507755-5ff4-48d9-99d0-716b69a0a190 service swift] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:46.550645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39507755-5ff4-48d9-99d0-716b69a0a190 service swift] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:46.550951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39507755-5ff4-48d9-99d0-716b69a0a190 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:46.562039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39507755-5ff4-48d9-99d0-716b69a0a190 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:46.564059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39507755-5ff4-48d9-99d0-716b69a0a190 service swift] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:46.574574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 882/1757] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:51:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 18:51:46.981216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:47.484060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:47.762223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d78f6-4d03-44ac-a3d1-3daf218d3b54 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:47.762511 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d78f6-4d03-44ac-a3d1-3daf218d3b54 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:47.762767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d78f6-4d03-44ac-a3d1-3daf218d3b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:47.818461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f48d78f6-4d03-44ac-a3d1-3daf218d3b54 None None] MFA Rules not processed for user `bcf30ef695154d3a97d7e589aca21233`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:47.822177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f48d78f6-4d03-44ac-a3d1-3daf218d3b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:47.872418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 876/1758] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Sep 8 18:51:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:47.934115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39507755-5ff4-48d9-99d0-716b69a0a190 service swift] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:47.939542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-39507755-5ff4-48d9-99d0-716b69a0a190 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:47.957107 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d78f6-4d03-44ac-a3d1-3daf218d3b54 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:47.962965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39507755-5ff4-48d9-99d0-716b69a0a190 service swift] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:47.972402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be0a69-4a13-4fb9-bf94-ee9b52d10392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:47.973063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be0a69-4a13-4fb9-bf94-ee9b52d10392 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:47.974978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be0a69-4a13-4fb9-bf94-ee9b52d10392 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:47.975196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be0a69-4a13-4fb9-bf94-ee9b52d10392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:47.983366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d78f6-4d03-44ac-a3d1-3daf218d3b54 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:47.989974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d128789-7738-4e4f-ab5a-6a65878076ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:47.990236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d128789-7738-4e4f-ab5a-6a65878076ab service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:47.990521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d128789-7738-4e4f-ab5a-6a65878076ab service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:47.990794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d128789-7738-4e4f-ab5a-6a65878076ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:48.004435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be0a69-4a13-4fb9-bf94-ee9b52d10392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:48.010396 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be0a69-4a13-4fb9-bf94-ee9b52d10392 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:48.015332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d128789-7738-4e4f-ab5a-6a65878076ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:48.015332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d128789-7738-4e4f-ab5a-6a65878076ab service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:48.030015 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 877/1759] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:48.036842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 883/1760] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:48.274073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d128789-7738-4e4f-ab5a-6a65878076ab service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:48.299984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d128789-7738-4e4f-ab5a-6a65878076ab service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:48.321800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:48.332010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:48.348740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:48.349364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] PATH_INFO: `/v3/users/05a00bd2c7614d50bf2d9c27f34a479f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:48.384343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05a00bd2c7614d50bf2d9c27f34a479f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:48.392791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:48.732261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:48.764402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05a00bd2c7614d50bf2d9c27f34a479f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:48.765329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05a00bd2c7614d50bf2d9c27f34a479f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:48.766112 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05a00bd2c7614d50bf2d9c27f34a479f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:48.799984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05a00bd2c7614d50bf2d9c27f34a479f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:48.807132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05a00bd2c7614d50bf2d9c27f34a479f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:48.955162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 878/1761] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:48 2022] DELETE /identity/v3/users/05a00bd2c7614d50bf2d9c27f34a479f => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:48.961334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9d4f-e6a2-4d33-9f4c-d22803165872 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:48.961887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9d4f-e6a2-4d33-9f4c-d22803165872 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:48.963289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9d4f-e6a2-4d33-9f4c-d22803165872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:49.055662 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c80a9d4f-e6a2-4d33-9f4c-d22803165872 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:49.062835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c80a9d4f-e6a2-4d33-9f4c-d22803165872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:49.168780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 884/1762] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:49.189301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:49.190893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:49.308504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853fd2e-1aea-453e-8ed0-b21f1d022012 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:49.313138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8efcf3-506c-4181-8c1d-ea5b7ba03a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:49.313829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8efcf3-506c-4181-8c1d-ea5b7ba03a0e service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:49.314140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8efcf3-506c-4181-8c1d-ea5b7ba03a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:49.314418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8efcf3-506c-4181-8c1d-ea5b7ba03a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:49.326341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8efcf3-506c-4181-8c1d-ea5b7ba03a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:49.327936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8efcf3-506c-4181-8c1d-ea5b7ba03a0e service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:49.340961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 879/1763] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:49.924351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:50.015774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9d4f-e6a2-4d33-9f4c-d22803165872 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:50.019688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c80a9d4f-e6a2-4d33-9f4c-d22803165872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:50.074118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9d4f-e6a2-4d33-9f4c-d22803165872 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:50.089035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:50.090209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:50.090651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:50.091084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] PATH_INFO: `/v3/projects/fa60d4abcb9443c09325824819dcec1a` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:50.093495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa60d4abcb9443c09325824819dcec1a)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:50.102580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:50.284337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:50.337449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa60d4abcb9443c09325824819dcec1a', 'request_id': 'req-c117aaf6-908f-4353-8ccc-3f755c3c18a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:50.347792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa60d4abcb9443c09325824819dcec1a', 'request_id': 'req-c117aaf6-908f-4353-8ccc-3f755c3c18a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:50.557812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa60d4abcb9443c09325824819dcec1a', 'request_id': 'req-c117aaf6-908f-4353-8ccc-3f755c3c18a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:50.571737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] The token cache is being invalidate because project fa60d4abcb9443c09325824819dcec1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:50.572298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:50.574338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c117aaf6-908f-4353-8ccc-3f755c3c18a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:50.579233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 885/1764] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:50 2022] DELETE /identity/v3/projects/fa60d4abcb9443c09325824819dcec1a => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:50.580415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8efcf3-506c-4181-8c1d-ea5b7ba03a0e service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:50.612593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8efcf3-506c-4181-8c1d-ea5b7ba03a0e service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:50.687239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:50.687917 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:50.688209 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:50.688688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] PATH_INFO: `/v3/users/fb979fdaa7c442458183ec52a642a248` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:50.690419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb979fdaa7c442458183ec52a642a248)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:50.693659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:50.886455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb979fdaa7c442458183ec52a642a248'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:50.887273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb979fdaa7c442458183ec52a642a248'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:50.888704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb979fdaa7c442458183ec52a642a248'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:50.901826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ee961-b359-4eca-a16a-b68e95bd081c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:50.902118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ee961-b359-4eca-a16a-b68e95bd081c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:50.902436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ee961-b359-4eca-a16a-b68e95bd081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:50.940640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb979fdaa7c442458183ec52a642a248'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:50.953814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1f4cdc46-ed9e-4a36-9214-6e301b1e651a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb979fdaa7c442458183ec52a642a248'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:50.972741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-317ee961-b359-4eca-a16a-b68e95bd081c None None] MFA Rules not processed for user `e1bf6afb94664ec1abc8a544241ba7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:50.974658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-317ee961-b359-4eca-a16a-b68e95bd081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.025990 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 880/1765] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:50 2022] DELETE /identity/v3/users/fb979fdaa7c442458183ec52a642a248 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:51.028661 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68932f-49e1-4295-affb-39bfb24aa13d None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.029503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68932f-49e1-4295-affb-39bfb24aa13d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.030011 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68932f-49e1-4295-affb-39bfb24aa13d None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.031844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 881/1766] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:51:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:51:51.040283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c0f3e-88f3-4ddf-9498-05a551bea962 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.041763 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c0f3e-88f3-4ddf-9498-05a551bea962 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.042981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c0f3e-88f3-4ddf-9498-05a551bea962 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.049346 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 882/1767] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:51:51 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:51:51.050038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 886/1768] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:50 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:51.060637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c0f3e-88f3-4ddf-9498-05a551bea962 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:51.060637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-818c0f3e-88f3-4ddf-9498-05a551bea962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.060859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dbb33-fb24-4c7c-bdc1-1b07c88e1222 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.060859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dbb33-fb24-4c7c-bdc1-1b07c88e1222 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.060859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dbb33-fb24-4c7c-bdc1-1b07c88e1222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.122006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-115dbb33-fb24-4c7c-bdc1-1b07c88e1222 None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:51.124044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-115dbb33-fb24-4c7c-bdc1-1b07c88e1222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.159848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c0f3e-88f3-4ddf-9498-05a551bea962 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:51.170702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a1600-920e-42d8-ac12-1c0debac48cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:51.171334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a1600-920e-42d8-ac12-1c0debac48cb admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.171614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a1600-920e-42d8-ac12-1c0debac48cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.171884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a1600-920e-42d8-ac12-1c0debac48cb admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.172513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a1600-920e-42d8-ac12-1c0debac48cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:51.174443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a1600-920e-42d8-ac12-1c0debac48cb admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:51.201382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 887/1769] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:51:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:51.202861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deea73-c1a6-47b2-a6f7-2f1eee45252b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.203156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deea73-c1a6-47b2-a6f7-2f1eee45252b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.203432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deea73-c1a6-47b2-a6f7-2f1eee45252b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.222944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-08deea73-c1a6-47b2-a6f7-2f1eee45252b None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:51.224145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-08deea73-c1a6-47b2-a6f7-2f1eee45252b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.233735 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 888/1770] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:51:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:51.234433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08deea73-c1a6-47b2-a6f7-2f1eee45252b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:51.236007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-08deea73-c1a6-47b2-a6f7-2f1eee45252b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.250104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08deea73-c1a6-47b2-a6f7-2f1eee45252b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:51.255007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:51.255634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.255913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.256183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] PATH_INFO: `/v3/users/0604b3cfeb664a33868f461f4b792156` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.257345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] RBAC: Authorizing `identity:delete_user(user_id=0604b3cfeb664a33868f461f4b792156)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:51.260045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:51.274151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 883/1771] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:51 2022] POST /identity/v3/projects => generated 403 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:51.274863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a1600-920e-42d8-ac12-1c0debac48cb admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:51.276463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4d8a1600-920e-42d8-ac12-1c0debac48cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.382617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a1600-920e-42d8-ac12-1c0debac48cb admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:51.389092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e460ba5-731d-469e-b968-1c78b41d08a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:51.393035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e460ba5-731d-469e-b968-1c78b41d08a5 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.393035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e460ba5-731d-469e-b968-1c78b41d08a5 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.393035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e460ba5-731d-469e-b968-1c78b41d08a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.400727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e460ba5-731d-469e-b968-1c78b41d08a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:51.402395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e460ba5-731d-469e-b968-1c78b41d08a5 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:51.414333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 884/1772] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:51.415594 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e460ba5-731d-469e-b968-1c78b41d08a5 service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:51.422041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1e460ba5-731d-469e-b968-1c78b41d08a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.442541 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e460ba5-731d-469e-b968-1c78b41d08a5 service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:51.449530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:51.455026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47549e69-084f-4cf9-bdfc-2531f17a12cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:51.456443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47549e69-084f-4cf9-bdfc-2531f17a12cf service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.456443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47549e69-084f-4cf9-bdfc-2531f17a12cf service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.456443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47549e69-084f-4cf9-bdfc-2531f17a12cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.468815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47549e69-084f-4cf9-bdfc-2531f17a12cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:51.471355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47549e69-084f-4cf9-bdfc-2531f17a12cf service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:51.482329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 885/1773] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:51.483322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47549e69-084f-4cf9-bdfc-2531f17a12cf service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:51.505457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0604b3cfeb664a33868f461f4b792156'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:51.506802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0604b3cfeb664a33868f461f4b792156'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:51.508264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0604b3cfeb664a33868f461f4b792156'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:51.520926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0604b3cfeb664a33868f461f4b792156'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:51.543120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0604b3cfeb664a33868f461f4b792156'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:51.557382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47549e69-084f-4cf9-bdfc-2531f17a12cf service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:51.562443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b8668-ae06-406b-866a-d2f62690b949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:51.571066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b8668-ae06-406b-866a-d2f62690b949 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.571066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b8668-ae06-406b-866a-d2f62690b949 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.571066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b8668-ae06-406b-866a-d2f62690b949 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.571066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b8668-ae06-406b-866a-d2f62690b949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:51.571066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b8668-ae06-406b-866a-d2f62690b949 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:51.657804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 889/1774] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:51 2022] DELETE /identity/v3/users/0604b3cfeb664a33868f461f4b792156 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:51.658733 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:51.659115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:51.663389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.674559 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 886/1775] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:51 2022] POST /identity/v3/users => generated 308 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:51.676180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cc8f5-e2fa-4f6a-8c8a-447066930de0 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.676468 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cc8f5-e2fa-4f6a-8c8a-447066930de0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.676743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cc8f5-e2fa-4f6a-8c8a-447066930de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.699156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-783cc8f5-e2fa-4f6a-8c8a-447066930de0 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:51.700322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-783cc8f5-e2fa-4f6a-8c8a-447066930de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.746307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32736b5f-15c9-4ca9-a7c2-21086fb769de admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:51.751509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:51.752258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.752605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.752957 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] PATH_INFO: `/v3/projects/033ff712076744e7b1313eee01d23692` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.754269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=033ff712076744e7b1313eee01d23692)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:51.756271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:51.771350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 887/1776] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:51.772466 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cc8f5-e2fa-4f6a-8c8a-447066930de0 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:51.774286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-783cc8f5-e2fa-4f6a-8c8a-447066930de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.787297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cc8f5-e2fa-4f6a-8c8a-447066930de0 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:51.792811 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed5880-ec41-475c-a2ba-2d8250f36c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:51.793417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5880-ec41-475c-a2ba-2d8250f36c62 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.793863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5880-ec41-475c-a2ba-2d8250f36c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.794158 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5880-ec41-475c-a2ba-2d8250f36c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.794811 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed5880-ec41-475c-a2ba-2d8250f36c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:51.796418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed5880-ec41-475c-a2ba-2d8250f36c62 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:51.808571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 888/1777] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:51.809823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed5880-ec41-475c-a2ba-2d8250f36c62 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:51.812355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-dbed5880-ec41-475c-a2ba-2d8250f36c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:51.892042 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed5880-ec41-475c-a2ba-2d8250f36c62 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:51.892820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033ff712076744e7b1313eee01d23692', 'request_id': 'req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:51.901010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033ff712076744e7b1313eee01d23692', 'request_id': 'req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:51.951431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a21bc-f198-43aa-9b78-8afeb6238e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:51.952097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a21bc-f198-43aa-9b78-8afeb6238e03 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:51.952381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a21bc-f198-43aa-9b78-8afeb6238e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:51.952662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a21bc-f198-43aa-9b78-8afeb6238e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:51.957561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033ff712076744e7b1313eee01d23692', 'request_id': 'req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:51.970601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] The token cache is being invalidate because project 033ff712076744e7b1313eee01d23692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:51.970996 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:51.971940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:51.974168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 890/1778] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:51 2022] DELETE /identity/v3/projects/033ff712076744e7b1313eee01d23692 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:51.975228 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:51.976743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:52.024677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a21bc-f198-43aa-9b78-8afeb6238e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:52.026189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a21bc-f198-43aa-9b78-8afeb6238e03 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:52.036152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 889/1779] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:52.037418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c18ec8-b8bb-436d-b5f5-54de2fd89590 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.037712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c18ec8-b8bb-436d-b5f5-54de2fd89590 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.037991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c18ec8-b8bb-436d-b5f5-54de2fd89590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.048334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfd0f9-9d5f-4268-86bb-1fef37e3a365 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:52.052773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1df70c-817f-47d9-98f3-3c00a969dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:52.053477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1df70c-817f-47d9-98f3-3c00a969dee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.053764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1df70c-817f-47d9-98f3-3c00a969dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.054094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1df70c-817f-47d9-98f3-3c00a969dee6 admin admin] PATH_INFO: `/v3/projects/b2704bcee2c745a2bdb3ca0731bb4e3e/users/375b64de23c04ab2aac16b3477b755b6/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.059761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-39c18ec8-b8bb-436d-b5f5-54de2fd89590 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:52.061023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-39c18ec8-b8bb-436d-b5f5-54de2fd89590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:52.069343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 890/1780] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:52.074142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1df70c-817f-47d9-98f3-3c00a969dee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2704bcee2c745a2bdb3ca0731bb4e3e, user_id=375b64de23c04ab2aac16b3477b755b6, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:52.077458 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be833c-a911-42fa-ad6f-2aff5952b999 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.077751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be833c-a911-42fa-ad6f-2aff5952b999 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.077873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1df70c-817f-47d9-98f3-3c00a969dee6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:52.078048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be833c-a911-42fa-ad6f-2aff5952b999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.098567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-33be833c-a911-42fa-ad6f-2aff5952b999 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:52.099776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-33be833c-a911-42fa-ad6f-2aff5952b999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:52.108793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 891/1781] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:52.113341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be833c-a911-42fa-ad6f-2aff5952b999 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:52.122616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be833c-a911-42fa-ad6f-2aff5952b999 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:52.127273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f54437-d134-49ef-b64a-d10016426e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:52.127871 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f54437-d134-49ef-b64a-d10016426e3e admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.128149 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f54437-d134-49ef-b64a-d10016426e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.128423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f54437-d134-49ef-b64a-d10016426e3e admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.129092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f54437-d134-49ef-b64a-d10016426e3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:52.129395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f54437-d134-49ef-b64a-d10016426e3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:52.131369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f54437-d134-49ef-b64a-d10016426e3e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:52.143645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 892/1782] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:51:52.147897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f54437-d134-49ef-b64a-d10016426e3e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:52.156150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 891/1783] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:51 2022] PUT /identity/v3/projects/b2704bcee2c745a2bdb3ca0731bb4e3e/users/375b64de23c04ab2aac16b3477b755b6/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:52.160115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1df70c-817f-47d9-98f3-3c00a969dee6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:52.160771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f54437-d134-49ef-b64a-d10016426e3e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:52.215622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:52.231823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1df70c-817f-47d9-98f3-3c00a969dee6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:52.234655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6444-6de1-4f7e-bdc3-c74d16777849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:52.246440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6444-6de1-4f7e-bdc3-c74d16777849 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.246440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6444-6de1-4f7e-bdc3-c74d16777849 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.246440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6444-6de1-4f7e-bdc3-c74d16777849 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.246440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c6444-6de1-4f7e-bdc3-c74d16777849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:52.246790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d73363-d494-43e0-ab3f-b0acec6d6090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:52.246790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73363-d494-43e0-ab3f-b0acec6d6090 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.246790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73363-d494-43e0-ab3f-b0acec6d6090 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.246790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73363-d494-43e0-ab3f-b0acec6d6090 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.247142 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c6444-6de1-4f7e-bdc3-c74d16777849 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:52.247306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d73363-d494-43e0-ab3f-b0acec6d6090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:52.247306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d73363-d494-43e0-ab3f-b0acec6d6090 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:52.254343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 892/1784] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:51:52.257125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d73363-d494-43e0-ab3f-b0acec6d6090 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:52.269843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:52.270762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d73363-d494-43e0-ab3f-b0acec6d6090 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:52.276509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55c8bb-ad9c-463a-bbb5-2ae454cf754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:52.277388 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c8bb-ad9c-463a-bbb5-2ae454cf754e admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.277671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c8bb-ad9c-463a-bbb5-2ae454cf754e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.277979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c8bb-ad9c-463a-bbb5-2ae454cf754e admin admin] PATH_INFO: `/v3/projects/b2704bcee2c745a2bdb3ca0731bb4e3e/users/375b64de23c04ab2aac16b3477b755b6/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.279937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55c8bb-ad9c-463a-bbb5-2ae454cf754e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2704bcee2c745a2bdb3ca0731bb4e3e, user_id=375b64de23c04ab2aac16b3477b755b6, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:52.282715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55c8bb-ad9c-463a-bbb5-2ae454cf754e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:52.293184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 893/1785] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:52 2022] PUT /identity/v3/projects/b2704bcee2c745a2bdb3ca0731bb4e3e/users/375b64de23c04ab2aac16b3477b755b6/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:52.395290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 893/1786] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:52 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:51:52.396467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55c8bb-ad9c-463a-bbb5-2ae454cf754e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:52.463863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55c8bb-ad9c-463a-bbb5-2ae454cf754e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:52.488008 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6444-6de1-4f7e-bdc3-c74d16777849 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:52.495982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5d9c6444-6de1-4f7e-bdc3-c74d16777849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:52.618681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60833e07-6063-42fa-a820-5514c296a27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:52.619492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60833e07-6063-42fa-a820-5514c296a27a admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.619844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60833e07-6063-42fa-a820-5514c296a27a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.620975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60833e07-6063-42fa-a820-5514c296a27a admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.621817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60833e07-6063-42fa-a820-5514c296a27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:52.624537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60833e07-6063-42fa-a820-5514c296a27a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:52.628695 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6444-6de1-4f7e-bdc3-c74d16777849 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:52.633519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:52.634237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.634517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.634792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] PATH_INFO: `/v3/projects/ea26513796cd49de8d3e1d8e724c53a5` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.635938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea26513796cd49de8d3e1d8e724c53a5)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:52.637928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:52.684579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:52.753310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 894/1787] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:52 2022] POST /identity/v3/users => generated 293 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:51:52.757275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60833e07-6063-42fa-a820-5514c296a27a admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:52.777934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60833e07-6063-42fa-a820-5514c296a27a admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:52.787153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48a71f-7a13-461c-96c7-e13099dbd39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:52.787793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48a71f-7a13-461c-96c7-e13099dbd39d admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.788075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48a71f-7a13-461c-96c7-e13099dbd39d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.788352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48a71f-7a13-461c-96c7-e13099dbd39d admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.789236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48a71f-7a13-461c-96c7-e13099dbd39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:52.791030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48a71f-7a13-461c-96c7-e13099dbd39d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:52.809603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 895/1788] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:52.810825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48a71f-7a13-461c-96c7-e13099dbd39d admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:52.826363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48a71f-7a13-461c-96c7-e13099dbd39d admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:52.832012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae39c5-2f9a-4685-954d-4d489ee03234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:52.832640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae39c5-2f9a-4685-954d-4d489ee03234 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:52.832909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae39c5-2f9a-4685-954d-4d489ee03234 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:52.833222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae39c5-2f9a-4685-954d-4d489ee03234 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:52.833869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae39c5-2f9a-4685-954d-4d489ee03234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:52.835463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae39c5-2f9a-4685-954d-4d489ee03234 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:52.851525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 896/1789] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:52.864149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae39c5-2f9a-4685-954d-4d489ee03234 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:52.891082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae39c5-2f9a-4685-954d-4d489ee03234 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:52.901065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea26513796cd49de8d3e1d8e724c53a5', 'request_id': 'req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:52.908922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea26513796cd49de8d3e1d8e724c53a5', 'request_id': 'req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:52.918463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea26513796cd49de8d3e1d8e724c53a5', 'request_id': 'req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:53.080247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79874953-093c-43d6-a7e7-0bb349c2c81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:53.082562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79874953-093c-43d6-a7e7-0bb349c2c81a admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:53.083014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79874953-093c-43d6-a7e7-0bb349c2c81a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:53.083491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79874953-093c-43d6-a7e7-0bb349c2c81a admin admin] PATH_INFO: `/v3/projects/ed1f4e6d6bfa4d2b9f9fadc8e3667362/users/ec20f3828ce349dbb788b13dcdacd18c/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:53.089161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] The token cache is being invalidate because project ea26513796cd49de8d3e1d8e724c53a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:53.094134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:53.094134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:53.094134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 894/1790] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:52 2022] DELETE /identity/v3/projects/ea26513796cd49de8d3e1d8e724c53a5 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:53.095876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:53.098790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:53.115187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b652c5-3d0c-47e5-95b4-04ff4b7350e2 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:53.116326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79874953-093c-43d6-a7e7-0bb349c2c81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1f4e6d6bfa4d2b9f9fadc8e3667362, user_id=ec20f3828ce349dbb788b13dcdacd18c, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:53.121814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:53.122883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:53.123340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:53.123648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] PATH_INFO: `/v3/users/4599045b62ae401fafeb902e17d3ac7a` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:53.125273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4599045b62ae401fafeb902e17d3ac7a)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:53.127330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:53.128318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79874953-093c-43d6-a7e7-0bb349c2c81a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:53.234150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 897/1791] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:52 2022] PUT /identity/v3/projects/ed1f4e6d6bfa4d2b9f9fadc8e3667362/users/ec20f3828ce349dbb788b13dcdacd18c/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:53.236991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295874e-83ed-4cc8-88b7-94cc31a70883 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:53.237408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295874e-83ed-4cc8-88b7-94cc31a70883 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:53.237827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295874e-83ed-4cc8-88b7-94cc31a70883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:53.300385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-5295874e-83ed-4cc8-88b7-94cc31a70883 None None] MFA Rules not processed for user `ec20f3828ce349dbb788b13dcdacd18c`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:53.302166 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5295874e-83ed-4cc8-88b7-94cc31a70883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:53.381693 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4599045b62ae401fafeb902e17d3ac7a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:53.383269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4599045b62ae401fafeb902e17d3ac7a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:53.384231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4599045b62ae401fafeb902e17d3ac7a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:53.421815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4599045b62ae401fafeb902e17d3ac7a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:53.423810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 898/1792] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:53.437411 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e76a48ec-6542-4b27-9185-0c58db12a5f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4599045b62ae401fafeb902e17d3ac7a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:53.450283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295874e-83ed-4cc8-88b7-94cc31a70883 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:53.454157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5295874e-83ed-4cc8-88b7-94cc31a70883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:53.510740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:53.569332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295874e-83ed-4cc8-88b7-94cc31a70883 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:53.574291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185774c-2a59-4d4c-b283-14dfcbcf2a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:53.575138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185774c-2a59-4d4c-b283-14dfcbcf2a1b service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:53.575489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185774c-2a59-4d4c-b283-14dfcbcf2a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:53.575767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185774c-2a59-4d4c-b283-14dfcbcf2a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:53.592884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185774c-2a59-4d4c-b283-14dfcbcf2a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:53.595466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185774c-2a59-4d4c-b283-14dfcbcf2a1b service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:53.629838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 899/1793] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:53.650076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185774c-2a59-4d4c-b283-14dfcbcf2a1b service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:53.651843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7185774c-2a59-4d4c-b283-14dfcbcf2a1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:53.749304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185774c-2a59-4d4c-b283-14dfcbcf2a1b service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:53.754694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:53.755554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:53.756054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:53.756482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] PATH_INFO: `/v3/users/ec20f3828ce349dbb788b13dcdacd18c` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:53.759566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec20f3828ce349dbb788b13dcdacd18c)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:53.763700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:53.765694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 895/1794] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:53 2022] DELETE /identity/v3/users/4599045b62ae401fafeb902e17d3ac7a => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:54.094885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec20f3828ce349dbb788b13dcdacd18c'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:54.095827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec20f3828ce349dbb788b13dcdacd18c'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:54.096880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec20f3828ce349dbb788b13dcdacd18c'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:54.106226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec20f3828ce349dbb788b13dcdacd18c'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:54.128454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec20f3828ce349dbb788b13dcdacd18c'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:54.219242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 900/1795] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:53 2022] DELETE /identity/v3/users/ec20f3828ce349dbb788b13dcdacd18c => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:51:54.225424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14079f98-c828-411b-bbe5-ed605c95ecba None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:54.225424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14079f98-c828-411b-bbe5-ed605c95ecba None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:54.225591 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14079f98-c828-411b-bbe5-ed605c95ecba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:54.267741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-14079f98-c828-411b-bbe5-ed605c95ecba None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:54.267741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-14079f98-c828-411b-bbe5-ed605c95ecba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:54.388224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 896/1796] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:54.401941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:54.403495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:54.517738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe93305-30ac-4666-aeda-dc0c173c41ff admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:54.521823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac7a24-1e12-4f2a-a82e-17f9def6dc77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:54.522821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac7a24-1e12-4f2a-a82e-17f9def6dc77 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:54.523286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac7a24-1e12-4f2a-a82e-17f9def6dc77 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:54.523960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac7a24-1e12-4f2a-a82e-17f9def6dc77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:54.534175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac7a24-1e12-4f2a-a82e-17f9def6dc77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:54.535918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac7a24-1e12-4f2a-a82e-17f9def6dc77 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:54.546882 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 901/1797] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:54.596799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798b9b7-f15b-4d5e-a894-7fbc9d7dfb13 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:54.596799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798b9b7-f15b-4d5e-a894-7fbc9d7dfb13 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:54.596799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798b9b7-f15b-4d5e-a894-7fbc9d7dfb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:54.639859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c798b9b7-f15b-4d5e-a894-7fbc9d7dfb13 None None] MFA Rules not processed for user `375b64de23c04ab2aac16b3477b755b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:54.642025 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c798b9b7-f15b-4d5e-a894-7fbc9d7dfb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:54.731568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 897/1798] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:54 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:54.824237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac7a24-1e12-4f2a-a82e-17f9def6dc77 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:54.827097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-94ac7a24-1e12-4f2a-a82e-17f9def6dc77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:54.840827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac7a24-1e12-4f2a-a82e-17f9def6dc77 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:54.846151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:54.847990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:54.848475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:54.848923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] PATH_INFO: `/v3/projects/445f54899a844bd7a7db08c1de9b26db` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:54.850621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=445f54899a844bd7a7db08c1de9b26db)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:54.853156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:54.966170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445f54899a844bd7a7db08c1de9b26db', 'request_id': 'req-ac785567-4ff0-4893-9637-e3844567ef4d'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:54.973033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445f54899a844bd7a7db08c1de9b26db', 'request_id': 'req-ac785567-4ff0-4893-9637-e3844567ef4d'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:55.091480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445f54899a844bd7a7db08c1de9b26db', 'request_id': 'req-ac785567-4ff0-4893-9637-e3844567ef4d'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:55.102951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] The token cache is being invalidate because project 445f54899a844bd7a7db08c1de9b26db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:55.103597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:55.103954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ac785567-4ff0-4893-9637-e3844567ef4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:55.105535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 902/1799] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:54 2022] DELETE /identity/v3/projects/445f54899a844bd7a7db08c1de9b26db => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:55.110742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa396c35-bbf5-4c44-a067-9e3a8477d468 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:55.111027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa396c35-bbf5-4c44-a067-9e3a8477d468 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:55.111304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa396c35-bbf5-4c44-a067-9e3a8477d468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:55.131594 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-aa396c35-bbf5-4c44-a067-9e3a8477d468 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:55.133748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-aa396c35-bbf5-4c44-a067-9e3a8477d468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:55.218747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 898/1800] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:55.226825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c87bcc-6460-4727-8a67-08e02bad9252 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:55.227344 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c87bcc-6460-4727-8a67-08e02bad9252 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:55.227828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c87bcc-6460-4727-8a67-08e02bad9252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:55.251208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a3c87bcc-6460-4727-8a67-08e02bad9252 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:55.252428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a3c87bcc-6460-4727-8a67-08e02bad9252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:55.265555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 903/1801] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:55.266703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa396c35-bbf5-4c44-a067-9e3a8477d468 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:55.285441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa396c35-bbf5-4c44-a067-9e3a8477d468 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:55.291666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e2ddb-2e37-47a3-93d5-ce076ef10f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:55.291925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e2ddb-2e37-47a3-93d5-ce076ef10f6b admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:55.292281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e2ddb-2e37-47a3-93d5-ce076ef10f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:55.292467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e2ddb-2e37-47a3-93d5-ce076ef10f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:55.302822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e2ddb-2e37-47a3-93d5-ce076ef10f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:55.303152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e2ddb-2e37-47a3-93d5-ce076ef10f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:55.305130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e2ddb-2e37-47a3-93d5-ce076ef10f6b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:55.336729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 899/1802] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:55.341138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c87bcc-6460-4727-8a67-08e02bad9252 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:55.350512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c87bcc-6460-4727-8a67-08e02bad9252 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:55.356167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42144c23-dce5-4662-a0d3-6b5eaab52c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:55.356770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42144c23-dce5-4662-a0d3-6b5eaab52c0e admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:55.357102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42144c23-dce5-4662-a0d3-6b5eaab52c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:55.357373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42144c23-dce5-4662-a0d3-6b5eaab52c0e admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:55.358008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42144c23-dce5-4662-a0d3-6b5eaab52c0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:55.360157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42144c23-dce5-4662-a0d3-6b5eaab52c0e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:55.486297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e2ddb-2e37-47a3-93d5-ce076ef10f6b admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:55.488011 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ce8e2ddb-2e37-47a3-93d5-ce076ef10f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:55.495135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 904/1803] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:55 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:55.512866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e2ddb-2e37-47a3-93d5-ce076ef10f6b admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:55.513569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42144c23-dce5-4662-a0d3-6b5eaab52c0e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:55.618675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42144c23-dce5-4662-a0d3-6b5eaab52c0e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:55.624668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:55.625501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:55.626343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:55.629579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] PATH_INFO: `/v3/projects/ed1f4e6d6bfa4d2b9f9fadc8e3667362` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:55.630612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1f4e6d6bfa4d2b9f9fadc8e3667362)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:55.633419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:55.634950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99331dfe-08b2-47ff-839a-04af00104b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:55.635859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99331dfe-08b2-47ff-839a-04af00104b0a admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:55.636351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99331dfe-08b2-47ff-839a-04af00104b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:55.636804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99331dfe-08b2-47ff-839a-04af00104b0a admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:55.637831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99331dfe-08b2-47ff-839a-04af00104b0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:55.641079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99331dfe-08b2-47ff-839a-04af00104b0a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:55.752182 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 905/1804] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:55 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:55.760033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd23d6-2054-4bf7-aefb-9398d6befcc4 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:55.760033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd23d6-2054-4bf7-aefb-9398d6befcc4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:55.760033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd23d6-2054-4bf7-aefb-9398d6befcc4 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:55.760033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 906/1805] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:51:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:51:55.763043 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd23d6-2054-4bf7-aefb-9398d6befcc4 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:55.810875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1f4e6d6bfa4d2b9f9fadc8e3667362', 'request_id': 'req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:55.828817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1f4e6d6bfa4d2b9f9fadc8e3667362', 'request_id': 'req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:55.840457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1f4e6d6bfa4d2b9f9fadc8e3667362', 'request_id': 'req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:55.918647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] The token cache is being invalidate because project ed1f4e6d6bfa4d2b9f9fadc8e3667362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:51:55.922175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:55.922285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd23d6-2054-4bf7-aefb-9398d6befcc4 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:55.922393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-77ed8c66-81b6-40f3-94a5-4b928f9fea0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:55.922759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 900/1806] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:55 2022] DELETE /identity/v3/projects/ed1f4e6d6bfa4d2b9f9fadc8e3667362 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:55.927564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b0038-44cf-4ef0-82bc-6757f5d1bc29 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:55.928041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b0038-44cf-4ef0-82bc-6757f5d1bc29 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:55.929116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b0038-44cf-4ef0-82bc-6757f5d1bc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:55.944279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230e57e-af2f-4c81-9c83-6836e0e912a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:55.944279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230e57e-af2f-4c81-9c83-6836e0e912a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:55.950501 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230e57e-af2f-4c81-9c83-6836e0e912a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:55.950501 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230e57e-af2f-4c81-9c83-6836e0e912a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:55.950501 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230e57e-af2f-4c81-9c83-6836e0e912a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:55.950501 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230e57e-af2f-4c81-9c83-6836e0e912a5 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:55.966918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 907/1807] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:55.966918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df907e1e-2a38-4477-a097-001051421946 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:55.966918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df907e1e-2a38-4477-a097-001051421946 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:55.973236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df907e1e-2a38-4477-a097-001051421946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:55.991181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-878b0038-44cf-4ef0-82bc-6757f5d1bc29 None None] MFA Rules not processed for user `a074714e365648b88c9d3a0495a17b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:55.995053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-878b0038-44cf-4ef0-82bc-6757f5d1bc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:56.003073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:56.015564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-df907e1e-2a38-4477-a097-001051421946 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:56.017448 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-df907e1e-2a38-4477-a097-001051421946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:56.032449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 908/1808] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:56.034170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df907e1e-2a38-4477-a097-001051421946 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:56.035642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-df907e1e-2a38-4477-a097-001051421946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:56.058321 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df907e1e-2a38-4477-a097-001051421946 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:56.097827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7113b-50ee-485d-95c1-b749a184b0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:56.098464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7113b-50ee-485d-95c1-b749a184b0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:56.098744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7113b-50ee-485d-95c1-b749a184b0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:56.099016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7113b-50ee-485d-95c1-b749a184b0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:56.099696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7113b-50ee-485d-95c1-b749a184b0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:56.108907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7113b-50ee-485d-95c1-b749a184b0d4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:56.126371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 909/1809] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:56.128019 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebc186-5797-4511-ae85-b3f4d98428f9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:56.128362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebc186-5797-4511-ae85-b3f4d98428f9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:56.128742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebc186-5797-4511-ae85-b3f4d98428f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:56.177198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1eebc186-5797-4511-ae85-b3f4d98428f9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:56.178913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1eebc186-5797-4511-ae85-b3f4d98428f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:56.184087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 901/1810] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:51:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:56.185192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b0038-44cf-4ef0-82bc-6757f5d1bc29 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:56.229144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 910/1811] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:56.234443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebc186-5797-4511-ae85-b3f4d98428f9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:56.237038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b0038-44cf-4ef0-82bc-6757f5d1bc29 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:56.237979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1eebc186-5797-4511-ae85-b3f4d98428f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:56.248597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b2ecc-57b7-4173-9e7a-6be53c986cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:56.250428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b2ecc-57b7-4173-9e7a-6be53c986cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:56.255479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b2ecc-57b7-4173-9e7a-6be53c986cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:56.256542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b2ecc-57b7-4173-9e7a-6be53c986cfb admin admin] PATH_INFO: `/v3/projects/2f5b472c306e442f8612d7fbfaf92015/users/a2dae6a9e8c1478282d707368ae5f7ec/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:56.267005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebc186-5797-4511-ae85-b3f4d98428f9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:56.273708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605d892-bf1c-4311-9644-187fcd67c685 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:56.274496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605d892-bf1c-4311-9644-187fcd67c685 service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:56.274879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605d892-bf1c-4311-9644-187fcd67c685 service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:56.275313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605d892-bf1c-4311-9644-187fcd67c685 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:56.288827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605d892-bf1c-4311-9644-187fcd67c685 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:56.290993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605d892-bf1c-4311-9644-187fcd67c685 service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:56.297162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b2ecc-57b7-4173-9e7a-6be53c986cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5b472c306e442f8612d7fbfaf92015, user_id=a2dae6a9e8c1478282d707368ae5f7ec, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:56.300140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b2ecc-57b7-4173-9e7a-6be53c986cfb admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:56.321404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 911/1812] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:56.322466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605d892-bf1c-4311-9644-187fcd67c685 service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:56.336575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605d892-bf1c-4311-9644-187fcd67c685 service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:56.362214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d14a2d-01e9-4c50-bf19-f6adb560a778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:56.362616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d14a2d-01e9-4c50-bf19-f6adb560a778 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:56.362907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d14a2d-01e9-4c50-bf19-f6adb560a778 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:56.363185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d14a2d-01e9-4c50-bf19-f6adb560a778 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:56.363827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d14a2d-01e9-4c50-bf19-f6adb560a778 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:51:56.364316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d14a2d-01e9-4c50-bf19-f6adb560a778 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:56.366109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d14a2d-01e9-4c50-bf19-f6adb560a778 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:56.395179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 912/1813] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:51:56.399991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d14a2d-01e9-4c50-bf19-f6adb560a778 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:56.424824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d14a2d-01e9-4c50-bf19-f6adb560a778 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:56.434455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 902/1814] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:56 2022] PUT /identity/v3/projects/2f5b472c306e442f8612d7fbfaf92015/users/a2dae6a9e8c1478282d707368ae5f7ec/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:56.434455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b2ecc-57b7-4173-9e7a-6be53c986cfb admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:56.439487 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0d9b2ecc-57b7-4173-9e7a-6be53c986cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:56.551580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1fd06-6540-4048-9e22-1bc6d647817b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:56.552525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1fd06-6540-4048-9e22-1bc6d647817b admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:56.553022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1fd06-6540-4048-9e22-1bc6d647817b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:56.553337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1fd06-6540-4048-9e22-1bc6d647817b admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:56.554018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1fd06-6540-4048-9e22-1bc6d647817b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:56.560726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1fd06-6540-4048-9e22-1bc6d647817b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:56.610468 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b2ecc-57b7-4173-9e7a-6be53c986cfb admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:56.617510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a0823-89e0-4d99-9621-c8ef18b54aea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:56.618312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a0823-89e0-4d99-9621-c8ef18b54aea service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:56.618736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a0823-89e0-4d99-9621-c8ef18b54aea service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:56.619494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a0823-89e0-4d99-9621-c8ef18b54aea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:56.621582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ef5a0823-89e0-4d99-9621-c8ef18b54aea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:56.711077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a0823-89e0-4d99-9621-c8ef18b54aea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:56.713036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a0823-89e0-4d99-9621-c8ef18b54aea service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:56.728313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 903/1815] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:56.729794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598eea7-e4cd-45e2-9387-a136dce0b865 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:56.730087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598eea7-e4cd-45e2-9387-a136dce0b865 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:56.730364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598eea7-e4cd-45e2-9387-a136dce0b865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:56.776165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2598eea7-e4cd-45e2-9387-a136dce0b865 None None] MFA Rules not processed for user `a2dae6a9e8c1478282d707368ae5f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:56.776409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 913/1816] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:56 2022] POST /identity/v3/projects => generated 371 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:56.780850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1fd06-6540-4048-9e22-1bc6d647817b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:56.782840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2598eea7-e4cd-45e2-9387-a136dce0b865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:56.936173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 904/1817] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:56.939723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598eea7-e4cd-45e2-9387-a136dce0b865 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:56.956803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1fd06-6540-4048-9e22-1bc6d647817b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:56.965663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598eea7-e4cd-45e2-9387-a136dce0b865 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:56.983984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f0e7a-6aad-45d4-b2fd-3dfd269df65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:56.985517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f0e7a-6aad-45d4-b2fd-3dfd269df65f admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:56.986091 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f0e7a-6aad-45d4-b2fd-3dfd269df65f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:56.986481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f0e7a-6aad-45d4-b2fd-3dfd269df65f admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:56.987686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f0e7a-6aad-45d4-b2fd-3dfd269df65f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:56.991482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f0e7a-6aad-45d4-b2fd-3dfd269df65f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:56.992876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e0ba7-b61c-4d18-9c2a-d5adad2d35bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:56.993781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e0ba7-b61c-4d18-9c2a-d5adad2d35bf admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:56.997345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e0ba7-b61c-4d18-9c2a-d5adad2d35bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:56.997874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e0ba7-b61c-4d18-9c2a-d5adad2d35bf admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:57.000251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e0ba7-b61c-4d18-9c2a-d5adad2d35bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:57.009985 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e0ba7-b61c-4d18-9c2a-d5adad2d35bf admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:57.212126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 905/1818] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:56 2022] POST /identity/v3/projects => generated 387 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:57.213088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f0e7a-6aad-45d4-b2fd-3dfd269df65f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:57.237830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 914/1819] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:56 2022] POST /identity/v3/users => generated 286 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:57.243112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e0ba7-b61c-4d18-9c2a-d5adad2d35bf admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:57.267330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e0ba7-b61c-4d18-9c2a-d5adad2d35bf admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:57.368058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:57.434679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f0e7a-6aad-45d4-b2fd-3dfd269df65f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:57.435105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e7744-3ce8-43a9-b5c2-b1476c80d397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:57.435233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320406c-d933-4401-b8d8-20b84a82a57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:57.435339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e7744-3ce8-43a9-b5c2-b1476c80d397 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:57.435427 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320406c-d933-4401-b8d8-20b84a82a57a admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:57.435512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e7744-3ce8-43a9-b5c2-b1476c80d397 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:57.435512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e7744-3ce8-43a9-b5c2-b1476c80d397 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:57.435512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e7744-3ce8-43a9-b5c2-b1476c80d397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:57.435512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e7744-3ce8-43a9-b5c2-b1476c80d397 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:57.435788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320406c-d933-4401-b8d8-20b84a82a57a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:57.435788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320406c-d933-4401-b8d8-20b84a82a57a admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:57.435788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320406c-d933-4401-b8d8-20b84a82a57a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:57.438424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320406c-d933-4401-b8d8-20b84a82a57a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:57.465073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 915/1820] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:51:57.479125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e7744-3ce8-43a9-b5c2-b1476c80d397 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:57.522665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e7744-3ce8-43a9-b5c2-b1476c80d397 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:57.539762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b36f6b-8859-4d89-978b-bba7eedb6379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:57.540573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b36f6b-8859-4d89-978b-bba7eedb6379 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:57.540895 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b36f6b-8859-4d89-978b-bba7eedb6379 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:57.543568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b36f6b-8859-4d89-978b-bba7eedb6379 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:57.549214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b36f6b-8859-4d89-978b-bba7eedb6379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:57.549214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b36f6b-8859-4d89-978b-bba7eedb6379 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:57.574128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 906/1821] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:57 2022] POST /identity/v3/users => generated 294 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:51:57.579291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320406c-d933-4401-b8d8-20b84a82a57a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:57.624885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320406c-d933-4401-b8d8-20b84a82a57a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:57.631405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 916/1822] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:57.642391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbcf01-7308-4218-953d-aace13136ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:57.643219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbcf01-7308-4218-953d-aace13136ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:57.643537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbcf01-7308-4218-953d-aace13136ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:57.643779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbcf01-7308-4218-953d-aace13136ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:57.644293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b36f6b-8859-4d89-978b-bba7eedb6379 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:57.644461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbcf01-7308-4218-953d-aace13136ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:57.669497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbcf01-7308-4218-953d-aace13136ad9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:57.673180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b36f6b-8859-4d89-978b-bba7eedb6379 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:57.680185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b07b3b-322b-440e-bfaf-e340d2b4ff5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:57.683241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b07b3b-322b-440e-bfaf-e340d2b4ff5f admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:57.683241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b07b3b-322b-440e-bfaf-e340d2b4ff5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:57.683241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b07b3b-322b-440e-bfaf-e340d2b4ff5f admin admin] PATH_INFO: `/v3/projects/005bff3532ff4a48b2b36a14e5b14ea9/users/e4d2eed6dab74b4bb0bfa969c4c4d9f0/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:57.702901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 907/1823] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:57.706173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbcf01-7308-4218-953d-aace13136ad9 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:57.723824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b07b3b-322b-440e-bfaf-e340d2b4ff5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=005bff3532ff4a48b2b36a14e5b14ea9, user_id=e4d2eed6dab74b4bb0bfa969c4c4d9f0, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:57.740475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b07b3b-322b-440e-bfaf-e340d2b4ff5f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:57.755129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbcf01-7308-4218-953d-aace13136ad9 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:57.801369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed70bf-496d-4d39-9652-50b79c494ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:57.806269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed70bf-496d-4d39-9652-50b79c494ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:57.807618 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed70bf-496d-4d39-9652-50b79c494ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:57.807908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed70bf-496d-4d39-9652-50b79c494ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:57.813790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed70bf-496d-4d39-9652-50b79c494ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:57.817360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed70bf-496d-4d39-9652-50b79c494ea4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:57.828407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 917/1824] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:57 2022] PUT /identity/v3/projects/005bff3532ff4a48b2b36a14e5b14ea9/users/e4d2eed6dab74b4bb0bfa969c4c4d9f0/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:57.829441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f4b34-adb1-45c1-861a-d48b7b98e6ee None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:57.829728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f4b34-adb1-45c1-861a-d48b7b98e6ee None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:57.830163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f4b34-adb1-45c1-861a-d48b7b98e6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:57.899370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-6a3f4b34-adb1-45c1-861a-d48b7b98e6ee None None] MFA Rules not processed for user `e4d2eed6dab74b4bb0bfa969c4c4d9f0`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:57.900796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6a3f4b34-adb1-45c1-861a-d48b7b98e6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:57.908117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 908/1825] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:57.910214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed70bf-496d-4d39-9652-50b79c494ea4 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:57.934144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed70bf-496d-4d39-9652-50b79c494ea4 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:58.073240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 918/1826] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:51:58.131425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea31c4b-8fa1-4cea-a7d1-785190eb4440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:58.131425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea31c4b-8fa1-4cea-a7d1-785190eb4440 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:58.131425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea31c4b-8fa1-4cea-a7d1-785190eb4440 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:58.131425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea31c4b-8fa1-4cea-a7d1-785190eb4440 admin admin] PATH_INFO: `/v3/projects/91f53c0a6e644e7fb32465581a88dd6c/users/f194b16f79b8455ab3767d799e3dd385/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:58.149196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f4b34-adb1-45c1-861a-d48b7b98e6ee None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:58.158649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6a3f4b34-adb1-45c1-861a-d48b7b98e6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:58.159136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea31c4b-8fa1-4cea-a7d1-785190eb4440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f53c0a6e644e7fb32465581a88dd6c, user_id=f194b16f79b8455ab3767d799e3dd385, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:58.162280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea31c4b-8fa1-4cea-a7d1-785190eb4440 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:58.294725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 909/1827] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:57 2022] PUT /identity/v3/projects/91f53c0a6e644e7fb32465581a88dd6c/users/f194b16f79b8455ab3767d799e3dd385/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:58.298851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f4b34-adb1-45c1-861a-d48b7b98e6ee None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:58.309633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4ff13-a448-4f95-a186-90a2ec288525 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:58.309991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4ff13-a448-4f95-a186-90a2ec288525 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:58.310459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4ff13-a448-4f95-a186-90a2ec288525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:58.369061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-bbc4ff13-a448-4f95-a186-90a2ec288525 None None] MFA Rules not processed for user `f194b16f79b8455ab3767d799e3dd385`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:58.372111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bbc4ff13-a448-4f95-a186-90a2ec288525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:58.435728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d94900-1de2-498f-8812-e68ad0f1d359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:58.436331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d94900-1de2-498f-8812-e68ad0f1d359 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:58.436647 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d94900-1de2-498f-8812-e68ad0f1d359 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:58.436940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d94900-1de2-498f-8812-e68ad0f1d359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:58.514880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 910/1828] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:51:58.519859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4ff13-a448-4f95-a186-90a2ec288525 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:58.600543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d94900-1de2-498f-8812-e68ad0f1d359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:58.603190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d94900-1de2-498f-8812-e68ad0f1d359 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:58.633479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 919/1829] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:51:58.668609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d94900-1de2-498f-8812-e68ad0f1d359 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:58.713122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4ff13-a448-4f95-a186-90a2ec288525 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:58.728360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7cf9a2-b3f2-45af-a098-cc05675da94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:58.730284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cf9a2-b3f2-45af-a098-cc05675da94f admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:58.731638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cf9a2-b3f2-45af-a098-cc05675da94f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:58.731937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cf9a2-b3f2-45af-a098-cc05675da94f admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:58.734137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7cf9a2-b3f2-45af-a098-cc05675da94f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:58.738548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7cf9a2-b3f2-45af-a098-cc05675da94f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:58.819197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d94900-1de2-498f-8812-e68ad0f1d359 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:58.825800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:58.827642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:58.827929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:58.828218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] PATH_INFO: `/v3/users/e4d2eed6dab74b4bb0bfa969c4c4d9f0` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:58.830403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d2eed6dab74b4bb0bfa969c4c4d9f0)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:58.833017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:58.905409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 911/1830] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:51:58 2022] POST /identity/v3/projects => generated 387 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:58.907564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82d319-6052-4916-83fb-11846afa8f36 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:58.907564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82d319-6052-4916-83fb-11846afa8f36 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:58.907564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82d319-6052-4916-83fb-11846afa8f36 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:58.913504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 912/1831] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:51:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:51:58.914797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82d319-6052-4916-83fb-11846afa8f36 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:59.017614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82d319-6052-4916-83fb-11846afa8f36 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:59.019176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d98fdd-da3c-4e4b-aba6-2787f28d2f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:59.019799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d98fdd-da3c-4e4b-aba6-2787f28d2f71 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.020087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d98fdd-da3c-4e4b-aba6-2787f28d2f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.020360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d98fdd-da3c-4e4b-aba6-2787f28d2f71 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.021187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d98fdd-da3c-4e4b-aba6-2787f28d2f71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:59.023210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d98fdd-da3c-4e4b-aba6-2787f28d2f71 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:59.245646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 913/1832] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:51:58 2022] POST /identity/v3/users => generated 300 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:51:59.247193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14086e45-00c9-4a48-9028-eb552d6cf003 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.247632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14086e45-00c9-4a48-9028-eb552d6cf003 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.247906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14086e45-00c9-4a48-9028-eb552d6cf003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.268097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-14086e45-00c9-4a48-9028-eb552d6cf003 None None] MFA Rules not processed for user `a074714e365648b88c9d3a0495a17b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:59.269270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-14086e45-00c9-4a48-9028-eb552d6cf003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:59.313955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d2eed6dab74b4bb0bfa969c4c4d9f0'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:59.316014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d2eed6dab74b4bb0bfa969c4c4d9f0'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:59.316767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d2eed6dab74b4bb0bfa969c4c4d9f0'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:59.324841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d2eed6dab74b4bb0bfa969c4c4d9f0'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:59.339317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d2eed6dab74b4bb0bfa969c4c4d9f0'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:59.350352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 914/1833] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:51:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:59.351276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14086e45-00c9-4a48-9028-eb552d6cf003 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:59.352956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-14086e45-00c9-4a48-9028-eb552d6cf003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:59.367595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14086e45-00c9-4a48-9028-eb552d6cf003 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:59.372684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:59.373485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.373834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.374179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] PATH_INFO: `/v3/users/e1bf6afb94664ec1abc8a544241ba7f3` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.375388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1bf6afb94664ec1abc8a544241ba7f3)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:59.377477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:59.443920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 920/1834] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:58 2022] DELETE /identity/v3/users/e4d2eed6dab74b4bb0bfa969c4c4d9f0 => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:51:59.444899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:59.446418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:59.544129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aedb52-59d2-49a9-bce7-6546efad71ef admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:59.548931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947eea-a8e2-4df9-8b95-4d59a54de67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:59.549609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947eea-a8e2-4df9-8b95-4d59a54de67a admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.549890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947eea-a8e2-4df9-8b95-4d59a54de67a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.550163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947eea-a8e2-4df9-8b95-4d59a54de67a admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.550806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5947eea-a8e2-4df9-8b95-4d59a54de67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:59.552535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5947eea-a8e2-4df9-8b95-4d59a54de67a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:59.567953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 921/1835] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:59.567953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683201d-43df-44b3-b1cb-85dfdeac75a5 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.567953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683201d-43df-44b3-b1cb-85dfdeac75a5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.567953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683201d-43df-44b3-b1cb-85dfdeac75a5 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.567953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 922/1836] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 8 18:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:51:59.568676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976e5dd-0880-4688-9b1f-f5daef922743 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.569120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976e5dd-0880-4688-9b1f-f5daef922743 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.569120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976e5dd-0880-4688-9b1f-f5daef922743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.583465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1bf6afb94664ec1abc8a544241ba7f3'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:59.584301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1bf6afb94664ec1abc8a544241ba7f3'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:59.585083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1bf6afb94664ec1abc8a544241ba7f3'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:59.593937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-5976e5dd-0880-4688-9b1f-f5daef922743 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:59.593937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5976e5dd-0880-4688-9b1f-f5daef922743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:59.600763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 923/1837] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:59.605841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976e5dd-0880-4688-9b1f-f5daef922743 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:59.607342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5976e5dd-0880-4688-9b1f-f5daef922743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:59.621533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1bf6afb94664ec1abc8a544241ba7f3'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:59.626440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976e5dd-0880-4688-9b1f-f5daef922743 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:59.627848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52cc63-6451-4a96-bc4a-eefa50dcefce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:59.628521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52cc63-6451-4a96-bc4a-eefa50dcefce admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.628803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52cc63-6451-4a96-bc4a-eefa50dcefce admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.629102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52cc63-6451-4a96-bc4a-eefa50dcefce admin admin] PATH_INFO: `/v3/projects/de6674bc3c6a431297648c6532266597/users/cfaa31e53f25403d8ceffab13d2266e2/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.630310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-304463a8-0b04-48e3-af2e-fe668395dd1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1bf6afb94664ec1abc8a544241ba7f3'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:51:59.645169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52cc63-6451-4a96-bc4a-eefa50dcefce admin admin] RBAC: Authorizing `identity:create_grant(project_id=de6674bc3c6a431297648c6532266597, user_id=cfaa31e53f25403d8ceffab13d2266e2, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:59.648427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52cc63-6451-4a96-bc4a-eefa50dcefce admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:59.706217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 915/1838] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:51:59 2022] DELETE /identity/v3/users/e1bf6afb94664ec1abc8a544241ba7f3 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:59.707558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb023e2-dc89-4c7d-acdb-f4bf805feb3e None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.707901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb023e2-dc89-4c7d-acdb-f4bf805feb3e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.708215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb023e2-dc89-4c7d-acdb-f4bf805feb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.726656 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:51:59.728644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-bfb023e2-dc89-4c7d-acdb-f4bf805feb3e None None] MFA Rules not processed for user `a074714e365648b88c9d3a0495a17b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:59.729899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bfb023e2-dc89-4c7d-acdb-f4bf805feb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:59.735374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 924/1839] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:59 2022] PUT /identity/v3/projects/de6674bc3c6a431297648c6532266597/users/cfaa31e53f25403d8ceffab13d2266e2/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:59.736446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52cc63-6451-4a96-bc4a-eefa50dcefce admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:59.738325 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ed52cc63-6451-4a96-bc4a-eefa50dcefce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:59.801506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 916/1840] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 8 18:51:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:51:59.802762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb023e2-dc89-4c7d-acdb-f4bf805feb3e None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:59.811852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52cc63-6451-4a96-bc4a-eefa50dcefce admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:59.817877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a415a-ff7f-4abb-ac1f-257d9910e6a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:59.819049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a415a-ff7f-4abb-ac1f-257d9910e6a6 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.819049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a415a-ff7f-4abb-ac1f-257d9910e6a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.819049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a415a-ff7f-4abb-ac1f-257d9910e6a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.877424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb023e2-dc89-4c7d-acdb-f4bf805feb3e None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:59.882047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709943d5-dc1d-464e-a80f-d567f72ad643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:59.883180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709943d5-dc1d-464e-a80f-d567f72ad643 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.883460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709943d5-dc1d-464e-a80f-d567f72ad643 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.883900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709943d5-dc1d-464e-a80f-d567f72ad643 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.884545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709943d5-dc1d-464e-a80f-d567f72ad643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:59.886043 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a415a-ff7f-4abb-ac1f-257d9910e6a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:59.886171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709943d5-dc1d-464e-a80f-d567f72ad643 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:59.887472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a415a-ff7f-4abb-ac1f-257d9910e6a6 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:59.896571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 917/1841] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:51:59.897700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709943d5-dc1d-464e-a80f-d567f72ad643 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:59.897833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 925/1842] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:59.901999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a415a-ff7f-4abb-ac1f-257d9910e6a6 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:59.914418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a415a-ff7f-4abb-ac1f-257d9910e6a6 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:59.914533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709943d5-dc1d-464e-a80f-d567f72ad643 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:59.918230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d820f5-e19b-4a7a-8402-29aa036e691c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:59.918423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bae9a8-d36e-4720-935f-0379142533dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:51:59.918900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d820f5-e19b-4a7a-8402-29aa036e691c admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.919152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bae9a8-d36e-4720-935f-0379142533dd service placement] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.919257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d820f5-e19b-4a7a-8402-29aa036e691c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.919435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d820f5-e19b-4a7a-8402-29aa036e691c admin admin] PATH_INFO: `/v3/projects/de6674bc3c6a431297648c6532266597/users/cfaa31e53f25403d8ceffab13d2266e2/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.919531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bae9a8-d36e-4720-935f-0379142533dd service placement] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.919706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bae9a8-d36e-4720-935f-0379142533dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.921468 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d820f5-e19b-4a7a-8402-29aa036e691c admin admin] RBAC: Authorizing `identity:create_grant(project_id=de6674bc3c6a431297648c6532266597, user_id=cfaa31e53f25403d8ceffab13d2266e2, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:59.921582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-31bae9a8-d36e-4720-935f-0379142533dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:59.924083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d820f5-e19b-4a7a-8402-29aa036e691c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:59.934552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bae9a8-d36e-4720-935f-0379142533dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:51:59.944138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 926/1843] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:51:59 2022] PUT /identity/v3/projects/de6674bc3c6a431297648c6532266597/users/cfaa31e53f25403d8ceffab13d2266e2/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:51:59.944528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bae9a8-d36e-4720-935f-0379142533dd service placement] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:51:59.947008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6912539-07b6-45e1-aa5f-1178d0d191dc None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:51:59.947297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6912539-07b6-45e1-aa5f-1178d0d191dc None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:51:59.947568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6912539-07b6-45e1-aa5f-1178d0d191dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:51:59.948732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 918/1844] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:51:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:51:59.959149 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bae9a8-d36e-4720-935f-0379142533dd service placement] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:51:59.961025 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-31bae9a8-d36e-4720-935f-0379142533dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:51:59.977718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bae9a8-d36e-4720-935f-0379142533dd service placement] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:51:59.984590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a6912539-07b6-45e1-aa5f-1178d0d191dc None None] MFA Rules not processed for user `cfaa31e53f25403d8ceffab13d2266e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:51:59.986252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a6912539-07b6-45e1-aa5f-1178d0d191dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:00.057766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:00.058359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:00.058840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:00.058965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] PATH_INFO: `/v3/projects/6f85922076484272833f13ba1f95862b` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:00.060126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f85922076484272833f13ba1f95862b)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:00.062110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:00.068662 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 927/1845] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:51:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:00.080152 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6912539-07b6-45e1-aa5f-1178d0d191dc None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:00.081867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a6912539-07b6-45e1-aa5f-1178d0d191dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:00.142165 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6912539-07b6-45e1-aa5f-1178d0d191dc None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:00.146533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840063a-600a-4113-ba1e-b76f98bc0b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:00.147186 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840063a-600a-4113-ba1e-b76f98bc0b76 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:00.147543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840063a-600a-4113-ba1e-b76f98bc0b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:00.148563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840063a-600a-4113-ba1e-b76f98bc0b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:00.162124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840063a-600a-4113-ba1e-b76f98bc0b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:00.165306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840063a-600a-4113-ba1e-b76f98bc0b76 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:00.176059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 928/1846] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:52:00.180332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f85922076484272833f13ba1f95862b', 'request_id': 'req-f774bd74-5bed-47dd-a420-b039d18d9981'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.188314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f85922076484272833f13ba1f95862b', 'request_id': 'req-f774bd74-5bed-47dd-a420-b039d18d9981'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.193977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f85922076484272833f13ba1f95862b', 'request_id': 'req-f774bd74-5bed-47dd-a420-b039d18d9981'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.322319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] The token cache is being invalidate because project 6f85922076484272833f13ba1f95862b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:00.322319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.322319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.322319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 919/1847] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:51:59 2022] DELETE /identity/v3/projects/6f85922076484272833f13ba1f95862b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:00.323053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840063a-600a-4113-ba1e-b76f98bc0b76 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:00.332611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840063a-600a-4113-ba1e-b76f98bc0b76 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:00.352317 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:00.358639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:00.414211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:00.415030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:00.415360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:00.415663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] PATH_INFO: `/v3/users/375b64de23c04ab2aac16b3477b755b6` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:00.416833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=375b64de23c04ab2aac16b3477b755b6)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:00.418794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:00.461031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774bd74-5bed-47dd-a420-b039d18d9981 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:00.465784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:00.473165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:00.473165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:00.473165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] PATH_INFO: `/v3/projects/005bff3532ff4a48b2b36a14e5b14ea9` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:00.473165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=005bff3532ff4a48b2b36a14e5b14ea9)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:00.473165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:00.569202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:00.626267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '375b64de23c04ab2aac16b3477b755b6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.626607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '005bff3532ff4a48b2b36a14e5b14ea9', 'request_id': 'req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.627069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '375b64de23c04ab2aac16b3477b755b6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.627688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '375b64de23c04ab2aac16b3477b755b6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.637343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '375b64de23c04ab2aac16b3477b755b6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.639031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '005bff3532ff4a48b2b36a14e5b14ea9', 'request_id': 'req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.648450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '005bff3532ff4a48b2b36a14e5b14ea9', 'request_id': 'req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.660958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '375b64de23c04ab2aac16b3477b755b6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.737394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] The token cache is being invalidate because project 005bff3532ff4a48b2b36a14e5b14ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:00.737897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.741109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3c9278d3-2a33-4ee7-9f5d-5bd575dd2847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:00.741109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 920/1848] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:00 2022] DELETE /identity/v3/projects/005bff3532ff4a48b2b36a14e5b14ea9 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:00.746185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c59682-a15c-4468-b4cc-66022f1704f8 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:00.746475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c59682-a15c-4468-b4cc-66022f1704f8 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:00.746751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c59682-a15c-4468-b4cc-66022f1704f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:00.781250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-55c59682-a15c-4468-b4cc-66022f1704f8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:00.782474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-55c59682-a15c-4468-b4cc-66022f1704f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:00.852498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 921/1849] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:00.861241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dae428-eb85-4175-8fe3-2fdb512dd2a0 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:00.861527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dae428-eb85-4175-8fe3-2fdb512dd2a0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:00.861802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dae428-eb85-4175-8fe3-2fdb512dd2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:00.880090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 929/1850] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:00 2022] DELETE /identity/v3/users/375b64de23c04ab2aac16b3477b755b6 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:00.882984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-91dae428-eb85-4175-8fe3-2fdb512dd2a0 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:00.884203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-91dae428-eb85-4175-8fe3-2fdb512dd2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:00.945564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 922/1851] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:00.949942 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:00.951351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:00.965016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fac951-cf0b-42fb-a913-5a701d4a2118 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:00.969811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d4ffa-9987-44f1-92bd-780107226bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:00.970437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d4ffa-9987-44f1-92bd-780107226bfa admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:00.970880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d4ffa-9987-44f1-92bd-780107226bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:00.971160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d4ffa-9987-44f1-92bd-780107226bfa admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:00.971813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d4ffa-9987-44f1-92bd-780107226bfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:00.972159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d4ffa-9987-44f1-92bd-780107226bfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:00.973756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d4ffa-9987-44f1-92bd-780107226bfa admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:00.986260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 930/1852] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:52:00.990296 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dae428-eb85-4175-8fe3-2fdb512dd2a0 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:00.999963 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dae428-eb85-4175-8fe3-2fdb512dd2a0 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:01.004542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70653845-109c-4c35-a81b-13b3337c0a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:01.005248 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70653845-109c-4c35-a81b-13b3337c0a91 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.005529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70653845-109c-4c35-a81b-13b3337c0a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.005804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70653845-109c-4c35-a81b-13b3337c0a91 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.006436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70653845-109c-4c35-a81b-13b3337c0a91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:01.008448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70653845-109c-4c35-a81b-13b3337c0a91 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:01.104324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 923/1853] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:00 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:01.108511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d4ffa-9987-44f1-92bd-780107226bfa admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:01.119968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d4ffa-9987-44f1-92bd-780107226bfa admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:01.140301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70653845-109c-4c35-a81b-13b3337c0a91 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:01.142183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-70653845-109c-4c35-a81b-13b3337c0a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:01.188593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fa668-ac81-4998-8463-3232319d3727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:01.189245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fa668-ac81-4998-8463-3232319d3727 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.189526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fa668-ac81-4998-8463-3232319d3727 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.189800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fa668-ac81-4998-8463-3232319d3727 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.190655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78fa668-ac81-4998-8463-3232319d3727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:01.192737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78fa668-ac81-4998-8463-3232319d3727 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:01.214840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70653845-109c-4c35-a81b-13b3337c0a91 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:01.219505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:01.221950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.221950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.221950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] PATH_INFO: `/v3/projects/b2704bcee2c745a2bdb3ca0731bb4e3e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.221950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2704bcee2c745a2bdb3ca0731bb4e3e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:01.223682 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:01.303602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 931/1854] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:01 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:01.305289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fa668-ac81-4998-8463-3232319d3727 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:01.315079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fa668-ac81-4998-8463-3232319d3727 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:01.319773 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bb6d6-3a85-4d2f-adda-611aefb2ea50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:01.320493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bb6d6-3a85-4d2f-adda-611aefb2ea50 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.325407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bb6d6-3a85-4d2f-adda-611aefb2ea50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.325407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bb6d6-3a85-4d2f-adda-611aefb2ea50 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.325407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bb6d6-3a85-4d2f-adda-611aefb2ea50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:01.325407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bb6d6-3a85-4d2f-adda-611aefb2ea50 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:01.334012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 932/1855] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:01.337829 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bb6d6-3a85-4d2f-adda-611aefb2ea50 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:01.348096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bb6d6-3a85-4d2f-adda-611aefb2ea50 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:01.354100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d9fc5-0ae8-4987-b55a-034edb28b604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:01.357794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d9fc5-0ae8-4987-b55a-034edb28b604 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.357794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d9fc5-0ae8-4987-b55a-034edb28b604 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.357794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d9fc5-0ae8-4987-b55a-034edb28b604 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.357794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d9fc5-0ae8-4987-b55a-034edb28b604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:01.357794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d9fc5-0ae8-4987-b55a-034edb28b604 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:01.372485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 933/1856] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:01.372673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2704bcee2c745a2bdb3ca0731bb4e3e', 'request_id': 'req-54a41f2e-a837-47f1-9396-32e996d8165d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:01.373555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d9fc5-0ae8-4987-b55a-034edb28b604 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:01.379359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2704bcee2c745a2bdb3ca0731bb4e3e', 'request_id': 'req-54a41f2e-a837-47f1-9396-32e996d8165d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:01.384021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d9fc5-0ae8-4987-b55a-034edb28b604 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:01.387895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2704bcee2c745a2bdb3ca0731bb4e3e', 'request_id': 'req-54a41f2e-a837-47f1-9396-32e996d8165d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:01.450466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806ea37-5c1d-46d8-97ae-560cf6bd8f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:01.451132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806ea37-5c1d-46d8-97ae-560cf6bd8f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.451411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806ea37-5c1d-46d8-97ae-560cf6bd8f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.451689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806ea37-5c1d-46d8-97ae-560cf6bd8f32 admin admin] PATH_INFO: `/v3/projects/fcd6521d51e8449b9d29a54ab3baf6c5/users/f03964cc3de642bfaa9ee5a4f05bfccc/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.469467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] The token cache is being invalidate because project b2704bcee2c745a2bdb3ca0731bb4e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:01.473470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:01.473470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:01.478232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806ea37-5c1d-46d8-97ae-560cf6bd8f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd6521d51e8449b9d29a54ab3baf6c5, user_id=f03964cc3de642bfaa9ee5a4f05bfccc, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:01.478393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 924/1857] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:01 2022] DELETE /identity/v3/projects/b2704bcee2c745a2bdb3ca0731bb4e3e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:01.478494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806ea37-5c1d-46d8-97ae-560cf6bd8f32 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:01.478581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:01.479304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:01.553655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a41f2e-a837-47f1-9396-32e996d8165d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:01.558554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3b27b-3aae-4ca9-b7ec-de77fa3fd204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:01.562616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3b27b-3aae-4ca9-b7ec-de77fa3fd204 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.562616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3b27b-3aae-4ca9-b7ec-de77fa3fd204 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.562616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3b27b-3aae-4ca9-b7ec-de77fa3fd204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.562616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-09a3b27b-3aae-4ca9-b7ec-de77fa3fd204 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:01.603113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 934/1858] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:01 2022] PUT /identity/v3/projects/fcd6521d51e8449b9d29a54ab3baf6c5/users/f03964cc3de642bfaa9ee5a4f05bfccc/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:01.604193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f72424-4ae0-4f03-9b61-52c08f25634d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.604447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f72424-4ae0-4f03-9b61-52c08f25634d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.604723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f72424-4ae0-4f03-9b61-52c08f25634d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.643332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-95f72424-4ae0-4f03-9b61-52c08f25634d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:01.644603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-95f72424-4ae0-4f03-9b61-52c08f25634d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:01.655796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3b27b-3aae-4ca9-b7ec-de77fa3fd204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:01.657804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3b27b-3aae-4ca9-b7ec-de77fa3fd204 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:01.671453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 925/1859] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:01.674030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4ca3c-f12e-4742-8e52-cf5c373313f5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.674544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4ca3c-f12e-4742-8e52-cf5c373313f5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.675120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4ca3c-f12e-4742-8e52-cf5c373313f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.734229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a2b4ca3c-f12e-4742-8e52-cf5c373313f5 None None] MFA Rules not processed for user `f03964cc3de642bfaa9ee5a4f05bfccc`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:01.735930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a2b4ca3c-f12e-4742-8e52-cf5c373313f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:01.760327 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 935/1860] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:01.783142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaee33-a382-4186-ae51-6d7c4f08925e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.784442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaee33-a382-4186-ae51-6d7c4f08925e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.784834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaee33-a382-4186-ae51-6d7c4f08925e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.853763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-88aaee33-a382-4186-ae51-6d7c4f08925e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:01.862504 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-88aaee33-a382-4186-ae51-6d7c4f08925e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:01.899153 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 926/1861] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:01.906196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 936/1862] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:01.910677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4ca3c-f12e-4742-8e52-cf5c373313f5 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:01.934255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4ca3c-f12e-4742-8e52-cf5c373313f5 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:01.945671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78aaa7-83c5-4469-91f9-5ba1154f011b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:01.946476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78aaa7-83c5-4469-91f9-5ba1154f011b admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:01.946664 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78aaa7-83c5-4469-91f9-5ba1154f011b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:01.947013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78aaa7-83c5-4469-91f9-5ba1154f011b admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:01.948529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78aaa7-83c5-4469-91f9-5ba1154f011b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:01.962674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78aaa7-83c5-4469-91f9-5ba1154f011b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:01.962674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78aaa7-83c5-4469-91f9-5ba1154f011b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:01.963073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aaee33-a382-4186-ae51-6d7c4f08925e None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:01.963073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-88aaee33-a382-4186-ae51-6d7c4f08925e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:01.993188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 927/1863] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:52:01.993188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78aaa7-83c5-4469-91f9-5ba1154f011b admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:02.004549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78aaa7-83c5-4469-91f9-5ba1154f011b admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:02.017490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd954b-feed-415a-85cd-d949edb52590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:02.018479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd954b-feed-415a-85cd-d949edb52590 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:02.018845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd954b-feed-415a-85cd-d949edb52590 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:02.019123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd954b-feed-415a-85cd-d949edb52590 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:02.019849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd954b-feed-415a-85cd-d949edb52590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:02.022378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd954b-feed-415a-85cd-d949edb52590 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:02.040210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aaee33-a382-4186-ae51-6d7c4f08925e None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:02.044369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e6315-8814-4504-9c8d-7c891db11870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:02.045143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e6315-8814-4504-9c8d-7c891db11870 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:02.048527 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e6315-8814-4504-9c8d-7c891db11870 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:02.048527 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e6315-8814-4504-9c8d-7c891db11870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:02.056148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e6315-8814-4504-9c8d-7c891db11870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:02.057976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e6315-8814-4504-9c8d-7c891db11870 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:02.068846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 937/1864] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:52:02.134034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 928/1865] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:01 2022] POST /identity/v3/projects => generated 357 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:02.152787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e6315-8814-4504-9c8d-7c891db11870 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:02.169996 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e6315-8814-4504-9c8d-7c891db11870 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:02.246566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68bf96-56b8-473a-a458-e30c41b5b2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:02.247458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68bf96-56b8-473a-a458-e30c41b5b2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:02.251281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68bf96-56b8-473a-a458-e30c41b5b2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:02.252312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68bf96-56b8-473a-a458-e30c41b5b2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:02.254974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68bf96-56b8-473a-a458-e30c41b5b2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:02.260119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68bf96-56b8-473a-a458-e30c41b5b2f1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:02.613547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 938/1866] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:02 2022] POST /identity/v3/users => generated 279 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:02.615657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd954b-feed-415a-85cd-d949edb52590 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:02.640595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd954b-feed-415a-85cd-d949edb52590 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:02.645199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273bdcf-2288-4fe8-9a7c-35f93615504e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:02.645802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273bdcf-2288-4fe8-9a7c-35f93615504e admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:02.646079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273bdcf-2288-4fe8-9a7c-35f93615504e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:02.646349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273bdcf-2288-4fe8-9a7c-35f93615504e admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:02.646994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273bdcf-2288-4fe8-9a7c-35f93615504e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:02.649402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273bdcf-2288-4fe8-9a7c-35f93615504e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:02.664302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 929/1867] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:02.714241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68bf96-56b8-473a-a458-e30c41b5b2f1 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:02.729386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68bf96-56b8-473a-a458-e30c41b5b2f1 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:02.742793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ecc6f-5421-4024-b3f8-5b40b4f61f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:02.743940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ecc6f-5421-4024-b3f8-5b40b4f61f67 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:02.744501 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ecc6f-5421-4024-b3f8-5b40b4f61f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:02.745180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ecc6f-5421-4024-b3f8-5b40b4f61f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:02.746491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ecc6f-5421-4024-b3f8-5b40b4f61f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:02.749646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ecc6f-5421-4024-b3f8-5b40b4f61f67 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:02.770171 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 939/1868] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:02.775641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273bdcf-2288-4fe8-9a7c-35f93615504e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:02.787177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273bdcf-2288-4fe8-9a7c-35f93615504e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:02.794168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d8058-15f1-46b0-b931-94b2fcd380d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:02.794857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d8058-15f1-46b0-b931-94b2fcd380d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:02.795274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d8058-15f1-46b0-b931-94b2fcd380d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:02.795425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d8058-15f1-46b0-b931-94b2fcd380d4 admin admin] PATH_INFO: `/v3/projects/e083988200c74ff5af3c543013549cd3/users/6227b1dd03864ccbb629eeb32f05d8e7/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:02.829998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d8058-15f1-46b0-b931-94b2fcd380d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e083988200c74ff5af3c543013549cd3, user_id=6227b1dd03864ccbb629eeb32f05d8e7, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:02.833508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d8058-15f1-46b0-b931-94b2fcd380d4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:02.934279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 930/1869] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:02 2022] PUT /identity/v3/projects/e083988200c74ff5af3c543013549cd3/users/6227b1dd03864ccbb629eeb32f05d8e7/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:02.947457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2907b59-07bf-4ec3-8985-a75d2090930b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:02.949361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2907b59-07bf-4ec3-8985-a75d2090930b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:02.949361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2907b59-07bf-4ec3-8985-a75d2090930b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:02.985768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:03.022413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a2907b59-07bf-4ec3-8985-a75d2090930b None None] MFA Rules not processed for user `6227b1dd03864ccbb629eeb32f05d8e7`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:03.024320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a2907b59-07bf-4ec3-8985-a75d2090930b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:03.031988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:03.152973 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 940/1870] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:03.260708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d8058-15f1-46b0-b931-94b2fcd380d4 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:03.426619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d8058-15f1-46b0-b931-94b2fcd380d4 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:03.436865 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b565de-8082-457b-90a7-dc0ffbfcc2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:03.438578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b565de-8082-457b-90a7-dc0ffbfcc2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:03.439168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b565de-8082-457b-90a7-dc0ffbfcc2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:03.439550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b565de-8082-457b-90a7-dc0ffbfcc2e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:03.440309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b565de-8082-457b-90a7-dc0ffbfcc2e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:03.442701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b565de-8082-457b-90a7-dc0ffbfcc2e8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:03.629181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 931/1871] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:03 2022] POST /identity/v3/projects => generated 355 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:03.638100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2907b59-07bf-4ec3-8985-a75d2090930b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:03.655181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2907b59-07bf-4ec3-8985-a75d2090930b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:03.824704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadaecde-7b56-46bd-aa2f-e8ac002f36c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:03.825492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadaecde-7b56-46bd-aa2f-e8ac002f36c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:03.825820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadaecde-7b56-46bd-aa2f-e8ac002f36c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:03.826167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadaecde-7b56-46bd-aa2f-e8ac002f36c4 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:03.826879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadaecde-7b56-46bd-aa2f-e8ac002f36c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:03.828871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadaecde-7b56-46bd-aa2f-e8ac002f36c4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:03.942436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 941/1872] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:03 2022] POST /identity/v3/users => generated 284 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:03.948252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b565de-8082-457b-90a7-dc0ffbfcc2e8 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:03.978986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b565de-8082-457b-90a7-dc0ffbfcc2e8 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:03.995932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca6ee3-c019-41ff-8172-91e21bd8ad28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:03.997032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca6ee3-c019-41ff-8172-91e21bd8ad28 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:03.997436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca6ee3-c019-41ff-8172-91e21bd8ad28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:03.997644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca6ee3-c019-41ff-8172-91e21bd8ad28 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:03.998487 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca6ee3-c019-41ff-8172-91e21bd8ad28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:04.000093 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca6ee3-c019-41ff-8172-91e21bd8ad28 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:04.044031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 932/1873] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:04.045260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadaecde-7b56-46bd-aa2f-e8ac002f36c4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:04.064410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadaecde-7b56-46bd-aa2f-e8ac002f36c4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:04.094282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ded80-1e01-4aa9-a2e4-2c4c52f13bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:04.095240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ded80-1e01-4aa9-a2e4-2c4c52f13bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:04.095637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ded80-1e01-4aa9-a2e4-2c4c52f13bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:04.096096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ded80-1e01-4aa9-a2e4-2c4c52f13bf8 admin admin] PATH_INFO: `/v3/projects/663f572aa9ff4073900815457db49595/users/e6a8a7865b5b43358ce54f944e14e9ea/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:04.147089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ded80-1e01-4aa9-a2e4-2c4c52f13bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=663f572aa9ff4073900815457db49595, user_id=e6a8a7865b5b43358ce54f944e14e9ea, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:04.163063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ded80-1e01-4aa9-a2e4-2c4c52f13bf8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:04.268032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 942/1874] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:04 2022] PUT /identity/v3/projects/663f572aa9ff4073900815457db49595/users/e6a8a7865b5b43358ce54f944e14e9ea/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:04.272375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca6ee3-c019-41ff-8172-91e21bd8ad28 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:04.284539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca6ee3-c019-41ff-8172-91e21bd8ad28 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:04.349895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b937f-4481-46b6-97cf-a75a3dff9ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:04.350516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b937f-4481-46b6-97cf-a75a3dff9ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:04.350804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b937f-4481-46b6-97cf-a75a3dff9ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:04.351184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b937f-4481-46b6-97cf-a75a3dff9ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:04.352218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b937f-4481-46b6-97cf-a75a3dff9ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:04.359284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b937f-4481-46b6-97cf-a75a3dff9ba0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:04.373260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 933/1875] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:04.377601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ded80-1e01-4aa9-a2e4-2c4c52f13bf8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:04.393221 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ded80-1e01-4aa9-a2e4-2c4c52f13bf8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:04.393507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff17a6-3a82-4e92-bd1f-cb7e2c2c7b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:04.394174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff17a6-3a82-4e92-bd1f-cb7e2c2c7b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:04.394459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff17a6-3a82-4e92-bd1f-cb7e2c2c7b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:04.394738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff17a6-3a82-4e92-bd1f-cb7e2c2c7b0f admin admin] PATH_INFO: `/v3/projects/663f572aa9ff4073900815457db49595/users/e6a8a7865b5b43358ce54f944e14e9ea/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:04.396676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff17a6-3a82-4e92-bd1f-cb7e2c2c7b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=663f572aa9ff4073900815457db49595, user_id=e6a8a7865b5b43358ce54f944e14e9ea, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:04.400018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff17a6-3a82-4e92-bd1f-cb7e2c2c7b0f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:04.411303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 943/1876] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:04 2022] PUT /identity/v3/projects/663f572aa9ff4073900815457db49595/users/e6a8a7865b5b43358ce54f944e14e9ea/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:04.422291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c37ef0-e78a-4f55-b6de-0612a6ac88b0 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:04.422608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c37ef0-e78a-4f55-b6de-0612a6ac88b0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:04.422959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c37ef0-e78a-4f55-b6de-0612a6ac88b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:04.458334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-82c37ef0-e78a-4f55-b6de-0612a6ac88b0 None None] MFA Rules not processed for user `e6a8a7865b5b43358ce54f944e14e9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:04.460199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-82c37ef0-e78a-4f55-b6de-0612a6ac88b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:04.597073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 934/1877] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:04.623641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff17a6-3a82-4e92-bd1f-cb7e2c2c7b0f admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:04.689511 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c37ef0-e78a-4f55-b6de-0612a6ac88b0 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:04.691400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-82c37ef0-e78a-4f55-b6de-0612a6ac88b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:04.712938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff17a6-3a82-4e92-bd1f-cb7e2c2c7b0f admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:04.718392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbe3d2-49fd-43cd-9ea1-e0c5b0e0eb6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:04.719035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbe3d2-49fd-43cd-9ea1-e0c5b0e0eb6b service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:04.719323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbe3d2-49fd-43cd-9ea1-e0c5b0e0eb6b service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:04.719596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbe3d2-49fd-43cd-9ea1-e0c5b0e0eb6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:04.734748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbe3d2-49fd-43cd-9ea1-e0c5b0e0eb6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:04.737975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbe3d2-49fd-43cd-9ea1-e0c5b0e0eb6b service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:04.754427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 944/1878] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:04.795171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c37ef0-e78a-4f55-b6de-0612a6ac88b0 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:04.801665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:04.802699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:04.802820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:04.803082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] PATH_INFO: `/v3/users/460e34193f8444078b79bddfb0dc85bd` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:04.805207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=460e34193f8444078b79bddfb0dc85bd)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:04.807992 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:05.051285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '460e34193f8444078b79bddfb0dc85bd'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:05.052339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '460e34193f8444078b79bddfb0dc85bd'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:05.052945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '460e34193f8444078b79bddfb0dc85bd'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:05.069453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '460e34193f8444078b79bddfb0dc85bd'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:05.080207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '460e34193f8444078b79bddfb0dc85bd'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:05.091879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:05.167263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:05.174258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 935/1879] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:04 2022] DELETE /identity/v3/users/460e34193f8444078b79bddfb0dc85bd => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:05.178909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f26647-82cc-437d-abd3-dcfc50c3806f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:05.180373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f26647-82cc-437d-abd3-dcfc50c3806f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:05.180772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f26647-82cc-437d-abd3-dcfc50c3806f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:05.191866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:05.204772 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:05.211000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e1f26647-82cc-437d-abd3-dcfc50c3806f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:05.212994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e1f26647-82cc-437d-abd3-dcfc50c3806f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:05.267700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a111f-19ac-403e-b2bb-962f37fdf491 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:05.272242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e374c1c-088f-4ed9-b699-6360c8b504bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:05.273519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e374c1c-088f-4ed9-b699-6360c8b504bb service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:05.273878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e374c1c-088f-4ed9-b699-6360c8b504bb service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:05.274221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e374c1c-088f-4ed9-b699-6360c8b504bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:05.276935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6e374c1c-088f-4ed9-b699-6360c8b504bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:05.293240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 945/1880] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:05.306407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f26647-82cc-437d-abd3-dcfc50c3806f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:05.308758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e1f26647-82cc-437d-abd3-dcfc50c3806f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:05.353597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e374c1c-088f-4ed9-b699-6360c8b504bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:05.355194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e374c1c-088f-4ed9-b699-6360c8b504bb service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:05.365977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 936/1881] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:05 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:52:05.392307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f26647-82cc-437d-abd3-dcfc50c3806f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:05.400938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04b237-f2e1-4654-acfd-e6475262ec01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:05.401851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04b237-f2e1-4654-acfd-e6475262ec01 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:05.402246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04b237-f2e1-4654-acfd-e6475262ec01 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:05.402618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04b237-f2e1-4654-acfd-e6475262ec01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:05.430574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04b237-f2e1-4654-acfd-e6475262ec01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:05.432102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04b237-f2e1-4654-acfd-e6475262ec01 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:05.443538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 946/1882] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:05.923017 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e374c1c-088f-4ed9-b699-6360c8b504bb service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:05.924629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6e374c1c-088f-4ed9-b699-6360c8b504bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:05.969442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e374c1c-088f-4ed9-b699-6360c8b504bb service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:05.984607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:05.985615 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:05.985955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:05.986167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] PATH_INFO: `/v3/projects/d23ba35e24794be29cf60e1edad2edd6` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:05.990680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23ba35e24794be29cf60e1edad2edd6)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:05.996416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:06.144039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23ba35e24794be29cf60e1edad2edd6', 'request_id': 'req-47aa0f40-17ab-4d30-a920-8ede64a69ef0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:06.150692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23ba35e24794be29cf60e1edad2edd6', 'request_id': 'req-47aa0f40-17ab-4d30-a920-8ede64a69ef0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:06.156226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23ba35e24794be29cf60e1edad2edd6', 'request_id': 'req-47aa0f40-17ab-4d30-a920-8ede64a69ef0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:06.240257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] The token cache is being invalidate because project d23ba35e24794be29cf60e1edad2edd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:06.240479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:06.241203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:06.243541 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 937/1883] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:05 2022] DELETE /identity/v3/projects/d23ba35e24794be29cf60e1edad2edd6 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:06.247797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04b237-f2e1-4654-acfd-e6475262ec01 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:06.258123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04b237-f2e1-4654-acfd-e6475262ec01 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:06.313548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:06.314213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:06.314493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:06.314767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] PATH_INFO: `/v3/users/ddee36d0ce64498c8481d8d3c7364c42` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:06.315959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddee36d0ce64498c8481d8d3c7364c42)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:06.318361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:06.547927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddee36d0ce64498c8481d8d3c7364c42'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:06.548782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddee36d0ce64498c8481d8d3c7364c42'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:06.549590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddee36d0ce64498c8481d8d3c7364c42'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:06.559225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddee36d0ce64498c8481d8d3c7364c42'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:06.581290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6a0ddb11-f293-4717-9fec-55f4538eb462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddee36d0ce64498c8481d8d3c7364c42'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:06.675288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 947/1884] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:06 2022] DELETE /identity/v3/users/ddee36d0ce64498c8481d8d3c7364c42 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:06.687301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:07.095534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:07.096959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:07.111539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa0f40-17ab-4d30-a920-8ede64a69ef0 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:07.171838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:07.172480 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:07.172744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:07.173019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] PATH_INFO: `/v3/projects/2ee18227c3a44a4c8fd009a35bdd2aad` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:07.174192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee18227c3a44a4c8fd009a35bdd2aad)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:07.176127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:07.195168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:07.293147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee18227c3a44a4c8fd009a35bdd2aad', 'request_id': 'req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:07.299804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee18227c3a44a4c8fd009a35bdd2aad', 'request_id': 'req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:07.306442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee18227c3a44a4c8fd009a35bdd2aad', 'request_id': 'req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:07.381874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] The token cache is being invalidate because project 2ee18227c3a44a4c8fd009a35bdd2aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:07.382195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:07.383394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8e3bbf1e-6066-4daa-851d-3670fc5bb95f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:07.384903 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 938/1885] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:07 2022] DELETE /identity/v3/projects/2ee18227c3a44a4c8fd009a35bdd2aad => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:07.390023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a953f0-fdf2-4ace-b668-9f2ec6c2fa90 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:07.390532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a953f0-fdf2-4ace-b668-9f2ec6c2fa90 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:07.390640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a953f0-fdf2-4ace-b668-9f2ec6c2fa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:07.410027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e1a953f0-fdf2-4ace-b668-9f2ec6c2fa90 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:07.411234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e1a953f0-fdf2-4ace-b668-9f2ec6c2fa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:07.469392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 948/1886] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:07.477959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede483b-bda5-42e6-a4ef-a6d603eae118 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:07.478241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede483b-bda5-42e6-a4ef-a6d603eae118 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:07.478516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede483b-bda5-42e6-a4ef-a6d603eae118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:07.500388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-cede483b-bda5-42e6-a4ef-a6d603eae118 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:07.501631 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cede483b-bda5-42e6-a4ef-a6d603eae118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:07.510725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 939/1887] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:07.516595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a953f0-fdf2-4ace-b668-9f2ec6c2fa90 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:07.526876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a953f0-fdf2-4ace-b668-9f2ec6c2fa90 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:07.531711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb7ef-f044-4f27-b720-1175ea3156ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:07.532474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb7ef-f044-4f27-b720-1175ea3156ad admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:07.532775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb7ef-f044-4f27-b720-1175ea3156ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:07.533104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb7ef-f044-4f27-b720-1175ea3156ad admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:07.533752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb7ef-f044-4f27-b720-1175ea3156ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:07.539079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb7ef-f044-4f27-b720-1175ea3156ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:07.539079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb7ef-f044-4f27-b720-1175ea3156ad admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:07.548289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 949/1888] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:52:07.553067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede483b-bda5-42e6-a4ef-a6d603eae118 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:07.562733 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede483b-bda5-42e6-a4ef-a6d603eae118 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:07.567565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc2b64-b133-4c99-87d5-a9fa4b881c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:07.568177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2b64-b133-4c99-87d5-a9fa4b881c5c admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:07.568460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2b64-b133-4c99-87d5-a9fa4b881c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:07.568734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2b64-b133-4c99-87d5-a9fa4b881c5c admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:07.569436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc2b64-b133-4c99-87d5-a9fa4b881c5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:07.571666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc2b64-b133-4c99-87d5-a9fa4b881c5c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:07.673247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 940/1889] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:07 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:07.677552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb7ef-f044-4f27-b720-1175ea3156ad admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:07.688302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb7ef-f044-4f27-b720-1175ea3156ad admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:07.765237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72662a-8a62-4716-9b8d-27bc375cdae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:07.765867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72662a-8a62-4716-9b8d-27bc375cdae4 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:07.766152 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72662a-8a62-4716-9b8d-27bc375cdae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:07.766851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72662a-8a62-4716-9b8d-27bc375cdae4 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:07.767450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72662a-8a62-4716-9b8d-27bc375cdae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:07.769665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72662a-8a62-4716-9b8d-27bc375cdae4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:07.861225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 950/1890] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:07 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:07.862400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc2b64-b133-4c99-87d5-a9fa4b881c5c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:07.873529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc2b64-b133-4c99-87d5-a9fa4b881c5c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:07.879045 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa82a9-1674-4838-92b7-9cb45a74192f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:07.884076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa82a9-1674-4838-92b7-9cb45a74192f admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:07.884076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa82a9-1674-4838-92b7-9cb45a74192f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:07.884076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa82a9-1674-4838-92b7-9cb45a74192f admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:07.884076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa82a9-1674-4838-92b7-9cb45a74192f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:07.884076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa82a9-1674-4838-92b7-9cb45a74192f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:07.896362 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 941/1891] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:07.900157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72662a-8a62-4716-9b8d-27bc375cdae4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:07.916083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72662a-8a62-4716-9b8d-27bc375cdae4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:07.922269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe46dfb-e443-422f-b6e3-75352d48e563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:07.923261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46dfb-e443-422f-b6e3-75352d48e563 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:07.923709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46dfb-e443-422f-b6e3-75352d48e563 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:07.924320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46dfb-e443-422f-b6e3-75352d48e563 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:07.925399 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe46dfb-e443-422f-b6e3-75352d48e563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:07.927145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe46dfb-e443-422f-b6e3-75352d48e563 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:07.954936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 951/1892] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:07.976794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa82a9-1674-4838-92b7-9cb45a74192f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:07.980359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa82a9-1674-4838-92b7-9cb45a74192f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:07.986979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd2eb2-c850-48bf-adfe-0c3525065c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:07.987652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd2eb2-c850-48bf-adfe-0c3525065c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:07.987928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd2eb2-c850-48bf-adfe-0c3525065c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:07.988201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd2eb2-c850-48bf-adfe-0c3525065c69 admin admin] PATH_INFO: `/v3/projects/9eb38aacaf9a47ae94a298488e9d634f/users/66646eb3aade431fb0752c8de8ce39a7/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:07.998571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:08.004233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd2eb2-c850-48bf-adfe-0c3525065c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb38aacaf9a47ae94a298488e9d634f, user_id=66646eb3aade431fb0752c8de8ce39a7, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:08.006924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd2eb2-c850-48bf-adfe-0c3525065c69 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:08.108797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:08.115220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 942/1893] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:07 2022] PUT /identity/v3/projects/9eb38aacaf9a47ae94a298488e9d634f/users/66646eb3aade431fb0752c8de8ce39a7/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:08.126920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d281e-0f46-47c0-9f06-9286fd30215f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:08.127303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d281e-0f46-47c0-9f06-9286fd30215f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:08.127592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d281e-0f46-47c0-9f06-9286fd30215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:08.157302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-898d281e-0f46-47c0-9f06-9286fd30215f None None] MFA Rules not processed for user `66646eb3aade431fb0752c8de8ce39a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:08.158522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-898d281e-0f46-47c0-9f06-9286fd30215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:08.222238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 952/1894] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:08.231751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd2eb2-c850-48bf-adfe-0c3525065c69 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:08.293764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd2eb2-c850-48bf-adfe-0c3525065c69 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:08.298744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9128-cdae-4d50-b255-1a52acdff41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:08.299432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9128-cdae-4d50-b255-1a52acdff41f admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:08.299712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9128-cdae-4d50-b255-1a52acdff41f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:08.299985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf9128-cdae-4d50-b255-1a52acdff41f admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:08.300696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf9128-cdae-4d50-b255-1a52acdff41f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:08.302663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf9128-cdae-4d50-b255-1a52acdff41f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:08.416827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 943/1895] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:08 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:08.421205 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d281e-0f46-47c0-9f06-9286fd30215f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:08.432642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d281e-0f46-47c0-9f06-9286fd30215f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:08.490791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffe4d8-2c0d-4680-9a65-d565f657a71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:08.491408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffe4d8-2c0d-4680-9a65-d565f657a71c admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:08.492404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffe4d8-2c0d-4680-9a65-d565f657a71c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:08.492781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffe4d8-2c0d-4680-9a65-d565f657a71c admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:08.493483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffe4d8-2c0d-4680-9a65-d565f657a71c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:08.499808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffe4d8-2c0d-4680-9a65-d565f657a71c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:08.588675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 953/1896] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:08 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:08.592863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9128-cdae-4d50-b255-1a52acdff41f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:08.605187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf9128-cdae-4d50-b255-1a52acdff41f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:08.611917 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32329e-f4c1-44f1-bea1-3d4e5408ee60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:08.612752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32329e-f4c1-44f1-bea1-3d4e5408ee60 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:08.613014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32329e-f4c1-44f1-bea1-3d4e5408ee60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:08.613393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32329e-f4c1-44f1-bea1-3d4e5408ee60 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:08.614110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32329e-f4c1-44f1-bea1-3d4e5408ee60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:08.615824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32329e-f4c1-44f1-bea1-3d4e5408ee60 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:08.634693 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 944/1897] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:08.635997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffe4d8-2c0d-4680-9a65-d565f657a71c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:08.646073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffe4d8-2c0d-4680-9a65-d565f657a71c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:08.650982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce386e-13b5-4146-93b4-fbde0de66888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:08.651636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce386e-13b5-4146-93b4-fbde0de66888 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:08.651917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce386e-13b5-4146-93b4-fbde0de66888 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:08.652194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce386e-13b5-4146-93b4-fbde0de66888 admin admin] PATH_INFO: `/v3/projects/3a3fd803859b4ef4af770eab0f64336f/users/f5ba066a5a8c4d80a6902b4f2aed8670/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:08.668587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce386e-13b5-4146-93b4-fbde0de66888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a3fd803859b4ef4af770eab0f64336f, user_id=f5ba066a5a8c4d80a6902b4f2aed8670, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:08.671245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce386e-13b5-4146-93b4-fbde0de66888 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:08.776831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 954/1898] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:08 2022] PUT /identity/v3/projects/3a3fd803859b4ef4af770eab0f64336f/users/f5ba066a5a8c4d80a6902b4f2aed8670/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:08.780786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32329e-f4c1-44f1-bea1-3d4e5408ee60 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:08.791838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32329e-f4c1-44f1-bea1-3d4e5408ee60 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:08.847387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0650a7-71a3-405c-9d4d-221e4e58ce96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:08.847921 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0650a7-71a3-405c-9d4d-221e4e58ce96 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:08.848202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0650a7-71a3-405c-9d4d-221e4e58ce96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:08.848484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0650a7-71a3-405c-9d4d-221e4e58ce96 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:08.849155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0650a7-71a3-405c-9d4d-221e4e58ce96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:08.850743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0650a7-71a3-405c-9d4d-221e4e58ce96 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:08.862653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 945/1899] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:08.867558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce386e-13b5-4146-93b4-fbde0de66888 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:08.877916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce386e-13b5-4146-93b4-fbde0de66888 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:08.882606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeae2ca-cab2-424a-97ae-94c55fd92ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:08.883244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeae2ca-cab2-424a-97ae-94c55fd92ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:08.883520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeae2ca-cab2-424a-97ae-94c55fd92ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:08.883789 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeae2ca-cab2-424a-97ae-94c55fd92ffb admin admin] PATH_INFO: `/v3/projects/3a3fd803859b4ef4af770eab0f64336f/users/f5ba066a5a8c4d80a6902b4f2aed8670/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:08.885698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeae2ca-cab2-424a-97ae-94c55fd92ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a3fd803859b4ef4af770eab0f64336f, user_id=f5ba066a5a8c4d80a6902b4f2aed8670, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:08.889215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeae2ca-cab2-424a-97ae-94c55fd92ffb admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:08.900712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 955/1900] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:08 2022] PUT /identity/v3/projects/3a3fd803859b4ef4af770eab0f64336f/users/f5ba066a5a8c4d80a6902b4f2aed8670/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:08.910964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a9540-dcf5-40f9-9f36-24be97d414eb None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:08.911252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a9540-dcf5-40f9-9f36-24be97d414eb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:08.911554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a9540-dcf5-40f9-9f36-24be97d414eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:08.941450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f05a9540-dcf5-40f9-9f36-24be97d414eb None None] MFA Rules not processed for user `f5ba066a5a8c4d80a6902b4f2aed8670`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:08.944332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f05a9540-dcf5-40f9-9f36-24be97d414eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:09.026347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 946/1901] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:09.034018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeae2ca-cab2-424a-97ae-94c55fd92ffb admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:09.035881 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ffeae2ca-cab2-424a-97ae-94c55fd92ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:09.120758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeae2ca-cab2-424a-97ae-94c55fd92ffb admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:09.125752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc71a8e-e6fe-4963-816f-68e161b2ef91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:09.126535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc71a8e-e6fe-4963-816f-68e161b2ef91 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:09.126930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc71a8e-e6fe-4963-816f-68e161b2ef91 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:09.127293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc71a8e-e6fe-4963-816f-68e161b2ef91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:09.138530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc71a8e-e6fe-4963-816f-68e161b2ef91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:09.140309 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc71a8e-e6fe-4963-816f-68e161b2ef91 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:09.150934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 956/1902] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:09.415931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a9540-dcf5-40f9-9f36-24be97d414eb None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:09.417884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f05a9540-dcf5-40f9-9f36-24be97d414eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:09.430948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a9540-dcf5-40f9-9f36-24be97d414eb None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:09.434740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb362d-fd03-4ad7-beb6-703ca7060e2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:09.435353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb362d-fd03-4ad7-beb6-703ca7060e2e service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:09.437236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb362d-fd03-4ad7-beb6-703ca7060e2e service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:09.437236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb362d-fd03-4ad7-beb6-703ca7060e2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:09.457729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:09.492978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:09.496851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb362d-fd03-4ad7-beb6-703ca7060e2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:09.498367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb362d-fd03-4ad7-beb6-703ca7060e2e service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:09.508532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 947/1903] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:10.195595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:10.309207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:10.919863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:10.948483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:11.290018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:11.314198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:11.502623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:11.548334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:11.691133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:11.719619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:11.814641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:12.005596 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:12.034054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:12.275659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:12.326053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:12.505343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc564a-5bcc-44b3-99ca-cccfade1e625 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:12.506241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc564a-5bcc-44b3-99ca-cccfade1e625 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:12.506929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc564a-5bcc-44b3-99ca-cccfade1e625 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:12.517953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 957/1904] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:52:12 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:52:12.523947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104cb4-f8a3-4ce9-a3a8-780df1ea540c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:12.524462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104cb4-f8a3-4ce9-a3a8-780df1ea540c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:12.524859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104cb4-f8a3-4ce9-a3a8-780df1ea540c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:12.547367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-1c104cb4-f8a3-4ce9-a3a8-780df1ea540c None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:12.548824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1c104cb4-f8a3-4ce9-a3a8-780df1ea540c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:12.656880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 948/1905] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:52:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:12.670860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc564a-5bcc-44b3-99ca-cccfade1e625 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:12.672570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e9cc564a-5bcc-44b3-99ca-cccfade1e625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:12.690385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc564a-5bcc-44b3-99ca-cccfade1e625 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:12.696337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708628d1-4d49-4b92-958d-548b8f89c7d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:12.697080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708628d1-4d49-4b92-958d-548b8f89c7d7 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:12.698511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708628d1-4d49-4b92-958d-548b8f89c7d7 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:12.698959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708628d1-4d49-4b92-958d-548b8f89c7d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:12.709357 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708628d1-4d49-4b92-958d-548b8f89c7d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:12.710937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708628d1-4d49-4b92-958d-548b8f89c7d7 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:12.723058 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 958/1906] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:12.738315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:12.868259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:13.120843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:13.625463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c104cb4-f8a3-4ce9-a3a8-780df1ea540c None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:13.636127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c104cb4-f8a3-4ce9-a3a8-780df1ea540c None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:13.639763 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03933b0-c356-43ad-8543-89beec855f94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:13.640368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03933b0-c356-43ad-8543-89beec855f94 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:13.640651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03933b0-c356-43ad-8543-89beec855f94 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:13.640927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03933b0-c356-43ad-8543-89beec855f94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:13.643707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d03933b0-c356-43ad-8543-89beec855f94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:13.709291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03933b0-c356-43ad-8543-89beec855f94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:13.710728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03933b0-c356-43ad-8543-89beec855f94 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:13.719690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 949/1907] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:13.770917 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:14.016821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:23.205516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:24.037431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708628d1-4d49-4b92-958d-548b8f89c7d7 service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:24.062381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=39 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:24.108869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708628d1-4d49-4b92-958d-548b8f89c7d7 service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:24.113893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:24.115218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:24.115218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:24.115218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] PATH_INFO: `/v3/users/66646eb3aade431fb0752c8de8ce39a7` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:24.116243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66646eb3aade431fb0752c8de8ce39a7)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:24.118234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:24.305535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66646eb3aade431fb0752c8de8ce39a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:24.306200 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66646eb3aade431fb0752c8de8ce39a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:24.307752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66646eb3aade431fb0752c8de8ce39a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:24.315047 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66646eb3aade431fb0752c8de8ce39a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:24.329076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66646eb3aade431fb0752c8de8ce39a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:24.417551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 959/1908] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:24 2022] DELETE /identity/v3/users/66646eb3aade431fb0752c8de8ce39a7 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:24.425613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909302e-9ac8-4733-9e8c-a4d440f3ff30 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:24.425899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909302e-9ac8-4733-9e8c-a4d440f3ff30 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:24.426174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909302e-9ac8-4733-9e8c-a4d440f3ff30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:24.445881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f909302e-9ac8-4733-9e8c-a4d440f3ff30 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:24.447150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f909302e-9ac8-4733-9e8c-a4d440f3ff30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:24.508872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 950/1909] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:24.519026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:24.520421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:24.577527 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4abfb-ee06-489c-aab0-ef4bbadf4780 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:24.581157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cb302-c09a-4ce3-9a55-7ff58dc73a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:24.581764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cb302-c09a-4ce3-9a55-7ff58dc73a2d service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:24.582042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cb302-c09a-4ce3-9a55-7ff58dc73a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:24.582311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cb302-c09a-4ce3-9a55-7ff58dc73a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:24.591516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cb302-c09a-4ce3-9a55-7ff58dc73a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:24.592999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cb302-c09a-4ce3-9a55-7ff58dc73a2d service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:24.609274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 960/1910] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:24.825506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909302e-9ac8-4733-9e8c-a4d440f3ff30 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:24.827939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f909302e-9ac8-4733-9e8c-a4d440f3ff30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:24.843293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909302e-9ac8-4733-9e8c-a4d440f3ff30 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:24.847898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:24.848542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:24.848823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:24.849134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] PATH_INFO: `/v3/users/a2dae6a9e8c1478282d707368ae5f7ec` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:24.850276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2dae6a9e8c1478282d707368ae5f7ec)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:24.852208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:25.110548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cb302-c09a-4ce3-9a55-7ff58dc73a2d service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:25.112235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6c0cb302-c09a-4ce3-9a55-7ff58dc73a2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:25.148896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cb302-c09a-4ce3-9a55-7ff58dc73a2d service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:25.151327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2dae6a9e8c1478282d707368ae5f7ec'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.153391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2dae6a9e8c1478282d707368ae5f7ec'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.155502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2dae6a9e8c1478282d707368ae5f7ec'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.163459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:25.164126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:25.164450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:25.164730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] PATH_INFO: `/v3/projects/9eb38aacaf9a47ae94a298488e9d634f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:25.168851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb38aacaf9a47ae94a298488e9d634f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:25.170824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:25.176309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2dae6a9e8c1478282d707368ae5f7ec'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.189900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-62664755-7a22-49e7-b29d-fcf49124714b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2dae6a9e8c1478282d707368ae5f7ec'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.200408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:25.348324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 951/1911] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:24 2022] DELETE /identity/v3/users/a2dae6a9e8c1478282d707368ae5f7ec => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:25.352612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739e64f-c4eb-460f-b877-262ac9531a34 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:25.353857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739e64f-c4eb-460f-b877-262ac9531a34 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:25.354455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739e64f-c4eb-460f-b877-262ac9531a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:25.373369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb38aacaf9a47ae94a298488e9d634f', 'request_id': 'req-c4d8899c-186f-44f5-ba7a-d563cb641591'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.378605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-d739e64f-c4eb-460f-b877-262ac9531a34 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:25.386694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d739e64f-c4eb-460f-b877-262ac9531a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:25.388669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb38aacaf9a47ae94a298488e9d634f', 'request_id': 'req-c4d8899c-186f-44f5-ba7a-d563cb641591'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.448757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 952/1912] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:25.459470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb38aacaf9a47ae94a298488e9d634f', 'request_id': 'req-c4d8899c-186f-44f5-ba7a-d563cb641591'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.461767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739e64f-c4eb-460f-b877-262ac9531a34 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:25.464089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d739e64f-c4eb-460f-b877-262ac9531a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:25.500308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] The token cache is being invalidate because project 9eb38aacaf9a47ae94a298488e9d634f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:25.501587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.503494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.505624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:25.507193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 961/1913] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:25 2022] DELETE /identity/v3/projects/9eb38aacaf9a47ae94a298488e9d634f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:25.526761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:25.531587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:25.559697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8899c-186f-44f5-ba7a-d563cb641591 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:25.572806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:25.572806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:25.573375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:25.573668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] PATH_INFO: `/v3/users/f5ba066a5a8c4d80a6902b4f2aed8670` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:25.578704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739e64f-c4eb-460f-b877-262ac9531a34 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:25.581987 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ba066a5a8c4d80a6902b4f2aed8670)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:25.583964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:25.585616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad260874-ccf5-4118-90b9-0facf96cf4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:25.586308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad260874-ccf5-4118-90b9-0facf96cf4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:25.588081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad260874-ccf5-4118-90b9-0facf96cf4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:25.588081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad260874-ccf5-4118-90b9-0facf96cf4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:25.607548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ad260874-ccf5-4118-90b9-0facf96cf4b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:25.624992 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad260874-ccf5-4118-90b9-0facf96cf4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:25.627402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad260874-ccf5-4118-90b9-0facf96cf4b1 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:25.639248 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 953/1914] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:52:25.840730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ba066a5a8c4d80a6902b4f2aed8670'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.841793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ba066a5a8c4d80a6902b4f2aed8670'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.842352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ba066a5a8c4d80a6902b4f2aed8670'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.853843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ba066a5a8c4d80a6902b4f2aed8670'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.868838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ba066a5a8c4d80a6902b4f2aed8670'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:25.957552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 962/1915] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:25 2022] DELETE /identity/v3/users/f5ba066a5a8c4d80a6902b4f2aed8670 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:26.058748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad260874-ccf5-4118-90b9-0facf96cf4b1 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:26.063254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ad260874-ccf5-4118-90b9-0facf96cf4b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:26.082119 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad260874-ccf5-4118-90b9-0facf96cf4b1 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:26.147180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:26.147817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:26.148101 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:26.148373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] PATH_INFO: `/v3/projects/2f5b472c306e442f8612d7fbfaf92015` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:26.149656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f5b472c306e442f8612d7fbfaf92015)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:26.151587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:26.276392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f5b472c306e442f8612d7fbfaf92015', 'request_id': 'req-4d7acc6b-14c7-4397-a835-484b9967b58e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.284773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f5b472c306e442f8612d7fbfaf92015', 'request_id': 'req-4d7acc6b-14c7-4397-a835-484b9967b58e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.291376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f5b472c306e442f8612d7fbfaf92015', 'request_id': 'req-4d7acc6b-14c7-4397-a835-484b9967b58e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.397125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] The token cache is being invalidate because project 2f5b472c306e442f8612d7fbfaf92015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:26.397545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.398181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.400528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 954/1916] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:26 2022] DELETE /identity/v3/projects/2f5b472c306e442f8612d7fbfaf92015 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:26.404470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:26.405929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:26.423453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0568cc-423f-4124-816a-5b4193529ccf admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:26.488479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:26.489078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:26.505623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:26.505623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:26.505623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:26.505623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] PATH_INFO: `/v3/users/f194b16f79b8455ab3767d799e3dd385` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:26.510579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f194b16f79b8455ab3767d799e3dd385)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:26.512737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:26.527475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7acc6b-14c7-4397-a835-484b9967b58e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:26.549831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:26.550495 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:26.550785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:26.551062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] PATH_INFO: `/v3/projects/3a3fd803859b4ef4af770eab0f64336f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:26.552407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a3fd803859b4ef4af770eab0f64336f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:26.553970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:26.554335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:26.781520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f194b16f79b8455ab3767d799e3dd385'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.782486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f194b16f79b8455ab3767d799e3dd385'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.783330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f194b16f79b8455ab3767d799e3dd385'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.803626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f194b16f79b8455ab3767d799e3dd385'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.808491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:26.822416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a3fd803859b4ef4af770eab0f64336f', 'request_id': 'req-bfb31918-9d4a-4bba-9b34-54ca8b216531'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.834664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-38934de7-3622-4caf-8ba2-10a791622f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f194b16f79b8455ab3767d799e3dd385'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.840991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a3fd803859b4ef4af770eab0f64336f', 'request_id': 'req-bfb31918-9d4a-4bba-9b34-54ca8b216531'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.852041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a3fd803859b4ef4af770eab0f64336f', 'request_id': 'req-bfb31918-9d4a-4bba-9b34-54ca8b216531'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:26.943623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 963/1917] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:26 2022] DELETE /identity/v3/users/f194b16f79b8455ab3767d799e3dd385 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:27.022202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] The token cache is being invalidate because project 3a3fd803859b4ef4af770eab0f64336f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:27.022523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:27.023640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-bfb31918-9d4a-4bba-9b34-54ca8b216531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:27.026641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 955/1918] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:26 2022] DELETE /identity/v3/projects/3a3fd803859b4ef4af770eab0f64336f => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:27.031943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ff892-69be-49c9-9c63-f3bd500bf37b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.032231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ff892-69be-49c9-9c63-f3bd500bf37b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.032505 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ff892-69be-49c9-9c63-f3bd500bf37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.052686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-6e8ff892-69be-49c9-9c63-f3bd500bf37b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:27.053925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6e8ff892-69be-49c9-9c63-f3bd500bf37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:27.121920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 964/1919] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:27.130669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625fcda-2ff7-4fd7-92a4-e783a87a5905 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.130956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625fcda-2ff7-4fd7-92a4-e783a87a5905 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.131228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625fcda-2ff7-4fd7-92a4-e783a87a5905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.153257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e625fcda-2ff7-4fd7-92a4-e783a87a5905 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:27.154490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e625fcda-2ff7-4fd7-92a4-e783a87a5905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:27.163045 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 956/1920] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:27.168622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ff892-69be-49c9-9c63-f3bd500bf37b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:27.179426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ff892-69be-49c9-9c63-f3bd500bf37b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:27.184219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e3a26-d0b9-4d89-a1ba-3521800d2215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:27.184833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e3a26-d0b9-4d89-a1ba-3521800d2215 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.185146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e3a26-d0b9-4d89-a1ba-3521800d2215 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.185420 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e3a26-d0b9-4d89-a1ba-3521800d2215 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.186069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e3a26-d0b9-4d89-a1ba-3521800d2215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:27.186370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e3a26-d0b9-4d89-a1ba-3521800d2215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:27.187979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e3a26-d0b9-4d89-a1ba-3521800d2215 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:27.199995 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 965/1921] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:52:27.204580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625fcda-2ff7-4fd7-92a4-e783a87a5905 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:27.230243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625fcda-2ff7-4fd7-92a4-e783a87a5905 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:27.252369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bc304-9e95-4d4d-aced-938c99049f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:27.253012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bc304-9e95-4d4d-aced-938c99049f4e admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.253353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bc304-9e95-4d4d-aced-938c99049f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.253630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bc304-9e95-4d4d-aced-938c99049f4e admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.254282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bc304-9e95-4d4d-aced-938c99049f4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:27.256242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bc304-9e95-4d4d-aced-938c99049f4e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:27.387336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 957/1922] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:27 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:27.391605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e3a26-d0b9-4d89-a1ba-3521800d2215 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:27.402389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e3a26-d0b9-4d89-a1ba-3521800d2215 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:27.468292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2d10-90d1-497d-8229-e0ca8dea01f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:27.469642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2d10-90d1-497d-8229-e0ca8dea01f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.469917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2d10-90d1-497d-8229-e0ca8dea01f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.470399 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2d10-90d1-497d-8229-e0ca8dea01f1 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.471165 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de2d10-90d1-497d-8229-e0ca8dea01f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:27.473132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de2d10-90d1-497d-8229-e0ca8dea01f1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:27.490590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bc304-9e95-4d4d-aced-938c99049f4e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:27.492345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-303bc304-9e95-4d4d-aced-938c99049f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:27.517803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bc304-9e95-4d4d-aced-938c99049f4e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:27.523282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:27.524124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.525946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.526469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] PATH_INFO: `/v3/projects/91f53c0a6e644e7fb32465581a88dd6c` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.527851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f53c0a6e644e7fb32465581a88dd6c)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:27.531844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:27.575749 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 966/1923] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:27 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:27.581844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2d10-90d1-497d-8229-e0ca8dea01f1 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:27.597905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2d10-90d1-497d-8229-e0ca8dea01f1 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:27.602893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8baba7-eeb6-4295-97d0-bc34acc4270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:27.604068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8baba7-eeb6-4295-97d0-bc34acc4270c admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.604417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8baba7-eeb6-4295-97d0-bc34acc4270c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.604697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8baba7-eeb6-4295-97d0-bc34acc4270c admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.605390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8baba7-eeb6-4295-97d0-bc34acc4270c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:27.607203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8baba7-eeb6-4295-97d0-bc34acc4270c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:27.625637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 967/1924] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:27.629984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8baba7-eeb6-4295-97d0-bc34acc4270c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:27.636035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:27.641815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8baba7-eeb6-4295-97d0-bc34acc4270c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:27.646866 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99ef39-5019-496e-b6e6-28caa19b65a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:27.649079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99ef39-5019-496e-b6e6-28caa19b65a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.649079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99ef39-5019-496e-b6e6-28caa19b65a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.649079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99ef39-5019-496e-b6e6-28caa19b65a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.649079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99ef39-5019-496e-b6e6-28caa19b65a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:27.650975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99ef39-5019-496e-b6e6-28caa19b65a6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:27.661156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f53c0a6e644e7fb32465581a88dd6c', 'request_id': 'req-1fc930e9-c655-4b22-8775-ae3410aab7d4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:27.664526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 968/1925] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:27.665616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99ef39-5019-496e-b6e6-28caa19b65a6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:27.669370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f53c0a6e644e7fb32465581a88dd6c', 'request_id': 'req-1fc930e9-c655-4b22-8775-ae3410aab7d4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:27.675251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f53c0a6e644e7fb32465581a88dd6c', 'request_id': 'req-1fc930e9-c655-4b22-8775-ae3410aab7d4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:27.677104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99ef39-5019-496e-b6e6-28caa19b65a6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:27.739209 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] The token cache is being invalidate because project 91f53c0a6e644e7fb32465581a88dd6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:27.739514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:27.740815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:27.746749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 958/1926] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:27 2022] DELETE /identity/v3/projects/91f53c0a6e644e7fb32465581a88dd6c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:27.750598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:27.752117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:27.755629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a819a2-53a4-4c0c-9c3b-6bc49c0f103a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:27.756304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a819a2-53a4-4c0c-9c3b-6bc49c0f103a admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.756885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a819a2-53a4-4c0c-9c3b-6bc49c0f103a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.757169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a819a2-53a4-4c0c-9c3b-6bc49c0f103a admin admin] PATH_INFO: `/v3/projects/688fabe7d978462f89bc5be042ef1128/users/e164588320914d6695404512f37c4aa3/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.768367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc930e9-c655-4b22-8775-ae3410aab7d4 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:27.773734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:27.774369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.774653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.775560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] PATH_INFO: `/v3/users/cfaa31e53f25403d8ceffab13d2266e2` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.776957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfaa31e53f25403d8ceffab13d2266e2)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:27.778197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a819a2-53a4-4c0c-9c3b-6bc49c0f103a admin admin] RBAC: Authorizing `identity:create_grant(project_id=688fabe7d978462f89bc5be042ef1128, user_id=e164588320914d6695404512f37c4aa3, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:27.779057 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:27.783079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a819a2-53a4-4c0c-9c3b-6bc49c0f103a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:27.843712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 969/1927] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:27 2022] PUT /identity/v3/projects/688fabe7d978462f89bc5be042ef1128/users/e164588320914d6695404512f37c4aa3/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:27.854091 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2913890-c7a3-4190-952e-2843dff957bc None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:27.854393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2913890-c7a3-4190-952e-2843dff957bc None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:27.854673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2913890-c7a3-4190-952e-2843dff957bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:27.883339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f2913890-c7a3-4190-952e-2843dff957bc None None] MFA Rules not processed for user `e164588320914d6695404512f37c4aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:27.885760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f2913890-c7a3-4190-952e-2843dff957bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:27.962611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 970/1928] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:27.971874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2913890-c7a3-4190-952e-2843dff957bc None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:27.973447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f2913890-c7a3-4190-952e-2843dff957bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:27.979805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfaa31e53f25403d8ceffab13d2266e2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:27.980613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfaa31e53f25403d8ceffab13d2266e2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:27.981280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfaa31e53f25403d8ceffab13d2266e2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:27.996849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfaa31e53f25403d8ceffab13d2266e2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:28.005035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfaa31e53f25403d8ceffab13d2266e2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:28.009366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=75,keystone:DELETE=21,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:28.049258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2913890-c7a3-4190-952e-2843dff957bc None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:28.053965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cde40b-1f3f-481a-8536-c875e7a7f236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:28.054581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cde40b-1f3f-481a-8536-c875e7a7f236 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.054862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cde40b-1f3f-481a-8536-c875e7a7f236 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.055137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cde40b-1f3f-481a-8536-c875e7a7f236 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.055771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cde40b-1f3f-481a-8536-c875e7a7f236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:28.058065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cde40b-1f3f-481a-8536-c875e7a7f236 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:28.067630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 959/1929] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:27 2022] DELETE /identity/v3/users/cfaa31e53f25403d8ceffab13d2266e2 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:28.172673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 971/1930] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:27 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:28.176679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:28.178092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:28.250004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8419268-cfd6-49d3-8d11-2fb2f053f059 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:28.311606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796152c-a8ad-4434-9a29-f3a3b3d1bfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:28.312228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796152c-a8ad-4434-9a29-f3a3b3d1bfd2 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.312506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796152c-a8ad-4434-9a29-f3a3b3d1bfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.312774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796152c-a8ad-4434-9a29-f3a3b3d1bfd2 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.313467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796152c-a8ad-4434-9a29-f3a3b3d1bfd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:28.315399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796152c-a8ad-4434-9a29-f3a3b3d1bfd2 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:28.454735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 960/1931] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:28 2022] POST /identity/v3/users => generated 299 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:28.458847 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cde40b-1f3f-481a-8536-c875e7a7f236 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:28.469812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cde40b-1f3f-481a-8536-c875e7a7f236 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:28.474431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c9aff-2f32-4c0f-9634-208cc555dce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:28.475036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c9aff-2f32-4c0f-9634-208cc555dce1 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.475315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c9aff-2f32-4c0f-9634-208cc555dce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.475588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c9aff-2f32-4c0f-9634-208cc555dce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.476223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c9aff-2f32-4c0f-9634-208cc555dce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:28.477985 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c9aff-2f32-4c0f-9634-208cc555dce1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:28.490045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 972/1932] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:28.494538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796152c-a8ad-4434-9a29-f3a3b3d1bfd2 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:28.504799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796152c-a8ad-4434-9a29-f3a3b3d1bfd2 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:28.513127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43925fb6-713f-4a51-8f2a-a42dcef19903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:28.513516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c9aff-2f32-4c0f-9634-208cc555dce1 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:28.513736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43925fb6-713f-4a51-8f2a-a42dcef19903 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.514127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43925fb6-713f-4a51-8f2a-a42dcef19903 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.514516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43925fb6-713f-4a51-8f2a-a42dcef19903 admin admin] PATH_INFO: `/v3/projects/c8e5f2e677d44ee1a4a31a4118d93d37/users/37a51029dff141ed9ffa05d7c70d87d6/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.515318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d57c9aff-2f32-4c0f-9634-208cc555dce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:28.531702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43925fb6-713f-4a51-8f2a-a42dcef19903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e5f2e677d44ee1a4a31a4118d93d37, user_id=37a51029dff141ed9ffa05d7c70d87d6, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:28.534561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43925fb6-713f-4a51-8f2a-a42dcef19903 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:28.542375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c9aff-2f32-4c0f-9634-208cc555dce1 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:28.547577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:28.548209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.548493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.548770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] PATH_INFO: `/v3/projects/de6674bc3c6a431297648c6532266597` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.549993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=de6674bc3c6a431297648c6532266597)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:28.551875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:28.610195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 961/1933] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:28 2022] PUT /identity/v3/projects/c8e5f2e677d44ee1a4a31a4118d93d37/users/37a51029dff141ed9ffa05d7c70d87d6/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:28.614483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43925fb6-713f-4a51-8f2a-a42dcef19903 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:28.645297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de6674bc3c6a431297648c6532266597', 'request_id': 'req-1d872e00-dd4f-411a-be6b-5285bc887bcc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:28.652759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de6674bc3c6a431297648c6532266597', 'request_id': 'req-1d872e00-dd4f-411a-be6b-5285bc887bcc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:28.683128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43925fb6-713f-4a51-8f2a-a42dcef19903 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:28.722868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de6674bc3c6a431297648c6532266597', 'request_id': 'req-1d872e00-dd4f-411a-be6b-5285bc887bcc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:28.735129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] The token cache is being invalidate because project de6674bc3c6a431297648c6532266597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:28.735892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:28.736602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1d872e00-dd4f-411a-be6b-5285bc887bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:28.738162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 973/1934] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:28 2022] DELETE /identity/v3/projects/de6674bc3c6a431297648c6532266597 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:28.741262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c24d4-4b24-4f9a-89a1-18a4dfe8e8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:28.741915 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c24d4-4b24-4f9a-89a1-18a4dfe8e8ee admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.742201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c24d4-4b24-4f9a-89a1-18a4dfe8e8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.742481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c24d4-4b24-4f9a-89a1-18a4dfe8e8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.743131 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c24d4-4b24-4f9a-89a1-18a4dfe8e8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:28.744635 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adee4-c567-44a0-a0cd-97ba644bff67 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.744903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adee4-c567-44a0-a0cd-97ba644bff67 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.745224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7adee4-c567-44a0-a0cd-97ba644bff67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.745343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c24d4-4b24-4f9a-89a1-18a4dfe8e8ee admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:28.755969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 962/1935] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:28.760366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c24d4-4b24-4f9a-89a1-18a4dfe8e8ee admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:28.762294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-042c24d4-4b24-4f9a-89a1-18a4dfe8e8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:28.770842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4a7adee4-c567-44a0-a0cd-97ba644bff67 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:28.772051 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4a7adee4-c567-44a0-a0cd-97ba644bff67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:28.776552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c24d4-4b24-4f9a-89a1-18a4dfe8e8ee admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:28.780570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 974/1936] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:28.787342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed53082-f7f9-426a-80d6-6252eee7df6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:28.788799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85d4e0-1e5b-4e4b-b291-beba5ba1a5e7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.789114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85d4e0-1e5b-4e4b-b291-beba5ba1a5e7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.789780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85d4e0-1e5b-4e4b-b291-beba5ba1a5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.791166 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed53082-f7f9-426a-80d6-6252eee7df6d admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.791658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed53082-f7f9-426a-80d6-6252eee7df6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.791957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed53082-f7f9-426a-80d6-6252eee7df6d admin admin] PATH_INFO: `/v3/projects/c8e5f2e677d44ee1a4a31a4118d93d37/users/37a51029dff141ed9ffa05d7c70d87d6/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.794204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed53082-f7f9-426a-80d6-6252eee7df6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e5f2e677d44ee1a4a31a4118d93d37, user_id=37a51029dff141ed9ffa05d7c70d87d6, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:28.799720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed53082-f7f9-426a-80d6-6252eee7df6d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:28.809975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 963/1937] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:28 2022] PUT /identity/v3/projects/c8e5f2e677d44ee1a4a31a4118d93d37/users/37a51029dff141ed9ffa05d7c70d87d6/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:28.814097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1e85d4e0-1e5b-4e4b-b291-beba5ba1a5e7 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:28.815453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1e85d4e0-1e5b-4e4b-b291-beba5ba1a5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:28.822787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e42f2-2e3b-4b76-b896-e3d6324e50e7 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.822787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e42f2-2e3b-4b76-b896-e3d6324e50e7 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.822787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e42f2-2e3b-4b76-b896-e3d6324e50e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.839922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:28.849619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-4b4e42f2-2e3b-4b76-b896-e3d6324e50e7 None None] MFA Rules not processed for user `37a51029dff141ed9ffa05d7c70d87d6`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:28.850826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4b4e42f2-2e3b-4b76-b896-e3d6324e50e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:28.879409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 975/1938] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:28.883655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85d4e0-1e5b-4e4b-b291-beba5ba1a5e7 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:28.893325 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85d4e0-1e5b-4e4b-b291-beba5ba1a5e7 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:28.898125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289ea44-1b31-4b4b-9be5-a5eca5805f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:28.898788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289ea44-1b31-4b4b-9be5-a5eca5805f7b admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.899076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289ea44-1b31-4b4b-9be5-a5eca5805f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.899354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289ea44-1b31-4b4b-9be5-a5eca5805f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.900141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289ea44-1b31-4b4b-9be5-a5eca5805f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:28.900454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289ea44-1b31-4b4b-9be5-a5eca5805f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:28.902147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289ea44-1b31-4b4b-9be5-a5eca5805f7b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:28.913754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 976/1939] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:52:28.919033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289ea44-1b31-4b4b-9be5-a5eca5805f7b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:28.922611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 964/1940] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:28.928743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289ea44-1b31-4b4b-9be5-a5eca5805f7b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:28.933810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e42f2-2e3b-4b76-b896-e3d6324e50e7 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:28.933946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a8757-dd8a-4bd5-a542-3d1431335608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:28.934585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a8757-dd8a-4bd5-a542-3d1431335608 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:28.934932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a8757-dd8a-4bd5-a542-3d1431335608 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:28.935172 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4b4e42f2-2e3b-4b76-b896-e3d6324e50e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:28.935342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a8757-dd8a-4bd5-a542-3d1431335608 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:28.936041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a8757-dd8a-4bd5-a542-3d1431335608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:28.938072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a8757-dd8a-4bd5-a542-3d1431335608 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:28.957678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:28.996920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e42f2-2e3b-4b76-b896-e3d6324e50e7 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:29.000619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a8c17-a553-4f04-8a04-67b58c1b786b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:29.001236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a8c17-a553-4f04-8a04-67b58c1b786b service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:29.001518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a8c17-a553-4f04-8a04-67b58c1b786b service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:29.001786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a8c17-a553-4f04-8a04-67b58c1b786b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:29.003600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a96a8c17-a553-4f04-8a04-67b58c1b786b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:29.012575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 977/1941] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:28 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:29.016523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a8757-dd8a-4bd5-a542-3d1431335608 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:29.063909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a8c17-a553-4f04-8a04-67b58c1b786b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:29.065703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a8c17-a553-4f04-8a04-67b58c1b786b service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:29.076530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 965/1942] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:52:29.084760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a8757-dd8a-4bd5-a542-3d1431335608 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:29.089496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b77f76-3ee6-4ff2-9c8e-d1d8b890e6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:29.090114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b77f76-3ee6-4ff2-9c8e-d1d8b890e6dd admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:29.090392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b77f76-3ee6-4ff2-9c8e-d1d8b890e6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:29.090665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b77f76-3ee6-4ff2-9c8e-d1d8b890e6dd admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:29.091303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b77f76-3ee6-4ff2-9c8e-d1d8b890e6dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:29.093299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b77f76-3ee6-4ff2-9c8e-d1d8b890e6dd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:29.183593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:29.191290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 978/1943] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:29 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:29.195641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a8c17-a553-4f04-8a04-67b58c1b786b service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:29.207875 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a8c17-a553-4f04-8a04-67b58c1b786b service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:29.214332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12a8ca-365f-4fb7-9b5e-b4a1f1efba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:29.214981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12a8ca-365f-4fb7-9b5e-b4a1f1efba68 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:29.215258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12a8ca-365f-4fb7-9b5e-b4a1f1efba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:29.215534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12a8ca-365f-4fb7-9b5e-b4a1f1efba68 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:29.216198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12a8ca-365f-4fb7-9b5e-b4a1f1efba68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:29.218126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12a8ca-365f-4fb7-9b5e-b4a1f1efba68 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:29.234344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 966/1944] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:29.236020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b77f76-3ee6-4ff2-9c8e-d1d8b890e6dd admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:29.248110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b77f76-3ee6-4ff2-9c8e-d1d8b890e6dd admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:29.251449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:29.261339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6132d95-d969-4126-87cc-017cab5e66d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:29.261978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6132d95-d969-4126-87cc-017cab5e66d0 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:29.262256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6132d95-d969-4126-87cc-017cab5e66d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:29.262530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6132d95-d969-4126-87cc-017cab5e66d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:29.263454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6132d95-d969-4126-87cc-017cab5e66d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:29.265140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6132d95-d969-4126-87cc-017cab5e66d0 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:29.280865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 979/1945] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:29.282204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12a8ca-365f-4fb7-9b5e-b4a1f1efba68 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:29.293082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12a8ca-365f-4fb7-9b5e-b4a1f1efba68 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:29.297783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff71b2-5569-4450-ba8e-ed28e143edcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:29.298453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff71b2-5569-4450-ba8e-ed28e143edcf admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:29.298733 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff71b2-5569-4450-ba8e-ed28e143edcf admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:29.299004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff71b2-5569-4450-ba8e-ed28e143edcf admin admin] PATH_INFO: `/v3/projects/4e6f8fe12a6e4784bc03bcd7710c8b95/users/6ce3d74110524ea1b9d851659efbff6e/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:29.316444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff71b2-5569-4450-ba8e-ed28e143edcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6f8fe12a6e4784bc03bcd7710c8b95, user_id=6ce3d74110524ea1b9d851659efbff6e, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:29.319156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff71b2-5569-4450-ba8e-ed28e143edcf admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:29.405196 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 967/1946] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:29 2022] PUT /identity/v3/projects/4e6f8fe12a6e4784bc03bcd7710c8b95/users/6ce3d74110524ea1b9d851659efbff6e/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:29.418560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa31bd-5ee9-4177-ac26-1729186218eb None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:29.418849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa31bd-5ee9-4177-ac26-1729186218eb None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:29.419153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa31bd-5ee9-4177-ac26-1729186218eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:29.447111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e2fa31bd-5ee9-4177-ac26-1729186218eb None None] MFA Rules not processed for user `6ce3d74110524ea1b9d851659efbff6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:29.448489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e2fa31bd-5ee9-4177-ac26-1729186218eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:29.513449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 980/1947] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:29.518108 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff71b2-5569-4450-ba8e-ed28e143edcf admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:29.580280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff71b2-5569-4450-ba8e-ed28e143edcf admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:29.584939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a0a6-608d-443e-9671-ca511d9fc45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:29.585605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a0a6-608d-443e-9671-ca511d9fc45e admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:29.585897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a0a6-608d-443e-9671-ca511d9fc45e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:29.586147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98a0a6-608d-443e-9671-ca511d9fc45e admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:29.586778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98a0a6-608d-443e-9671-ca511d9fc45e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:29.589732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98a0a6-608d-443e-9671-ca511d9fc45e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:29.696128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 968/1948] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:29 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:29.703863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa31bd-5ee9-4177-ac26-1729186218eb None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:29.717830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa31bd-5ee9-4177-ac26-1729186218eb None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:29.778712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121afa85-8e3e-4af3-9ef0-f9ad49f5c1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:29.779256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121afa85-8e3e-4af3-9ef0-f9ad49f5c1e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:29.779537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121afa85-8e3e-4af3-9ef0-f9ad49f5c1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:29.779811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121afa85-8e3e-4af3-9ef0-f9ad49f5c1e7 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:29.780476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121afa85-8e3e-4af3-9ef0-f9ad49f5c1e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:29.782424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121afa85-8e3e-4af3-9ef0-f9ad49f5c1e7 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:29.905301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 981/1949] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:29 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:29.908978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a0a6-608d-443e-9671-ca511d9fc45e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:29.924829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98a0a6-608d-443e-9671-ca511d9fc45e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:29.932217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7b6c5-fbca-49fd-983e-9f1b0896942e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:29.932911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7b6c5-fbca-49fd-983e-9f1b0896942e admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:29.933374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7b6c5-fbca-49fd-983e-9f1b0896942e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:29.934435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7b6c5-fbca-49fd-983e-9f1b0896942e admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:29.934435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7b6c5-fbca-49fd-983e-9f1b0896942e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:29.936292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7b6c5-fbca-49fd-983e-9f1b0896942e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:29.955721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 969/1950] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:29.960521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121afa85-8e3e-4af3-9ef0-f9ad49f5c1e7 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:29.975998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121afa85-8e3e-4af3-9ef0-f9ad49f5c1e7 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:29.983236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb22e67-0a82-48d2-a417-eef529479044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:29.984061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb22e67-0a82-48d2-a417-eef529479044 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:29.984347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb22e67-0a82-48d2-a417-eef529479044 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:29.985826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb22e67-0a82-48d2-a417-eef529479044 admin admin] PATH_INFO: `/v3/projects/a6488bc18d1441c7bc824eefc4ee7462/users/1933a294609b49039dc023af4b3065de/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:30.001128 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb22e67-0a82-48d2-a417-eef529479044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6488bc18d1441c7bc824eefc4ee7462, user_id=1933a294609b49039dc023af4b3065de, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:30.003888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb22e67-0a82-48d2-a417-eef529479044 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:30.006208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:30.100833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 982/1951] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:29 2022] PUT /identity/v3/projects/a6488bc18d1441c7bc824eefc4ee7462/users/1933a294609b49039dc023af4b3065de/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:30.102228 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7b6c5-fbca-49fd-983e-9f1b0896942e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:30.118033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7b6c5-fbca-49fd-983e-9f1b0896942e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:30.192526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291622c9-b107-4ff1-98cd-a9c4797219ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:30.193237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291622c9-b107-4ff1-98cd-a9c4797219ef admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:30.193527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291622c9-b107-4ff1-98cd-a9c4797219ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:30.193809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291622c9-b107-4ff1-98cd-a9c4797219ef admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:30.194767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291622c9-b107-4ff1-98cd-a9c4797219ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:30.196639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291622c9-b107-4ff1-98cd-a9c4797219ef admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:30.210048 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 970/1952] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:30.214832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb22e67-0a82-48d2-a417-eef529479044 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:30.229416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb22e67-0a82-48d2-a417-eef529479044 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:30.234741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591115f-f5e2-469f-a9f8-c4823d186ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:30.235825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591115f-f5e2-469f-a9f8-c4823d186ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:30.236112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591115f-f5e2-469f-a9f8-c4823d186ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:30.238688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591115f-f5e2-469f-a9f8-c4823d186ad2 admin admin] PATH_INFO: `/v3/projects/a6488bc18d1441c7bc824eefc4ee7462/users/1933a294609b49039dc023af4b3065de/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:30.238688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591115f-f5e2-469f-a9f8-c4823d186ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6488bc18d1441c7bc824eefc4ee7462, user_id=1933a294609b49039dc023af4b3065de, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:30.241387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591115f-f5e2-469f-a9f8-c4823d186ad2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:30.253382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 983/1953] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:30 2022] PUT /identity/v3/projects/a6488bc18d1441c7bc824eefc4ee7462/users/1933a294609b49039dc023af4b3065de/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:30.264104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14785a21-b970-4145-926b-3a94830a0e5d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:30.264395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14785a21-b970-4145-926b-3a94830a0e5d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:30.267859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14785a21-b970-4145-926b-3a94830a0e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:30.297952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:30.297952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-14785a21-b970-4145-926b-3a94830a0e5d None None] MFA Rules not processed for user `1933a294609b49039dc023af4b3065de`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:30.297952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-14785a21-b970-4145-926b-3a94830a0e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:30.395908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 971/1954] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:30.584736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591115f-f5e2-469f-a9f8-c4823d186ad2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:30.685529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591115f-f5e2-469f-a9f8-c4823d186ad2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:30.690339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e472d-2420-4536-a566-6ea525bd1253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:30.690971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e472d-2420-4536-a566-6ea525bd1253 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:30.691248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e472d-2420-4536-a566-6ea525bd1253 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:30.691519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e472d-2420-4536-a566-6ea525bd1253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:30.708944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e472d-2420-4536-a566-6ea525bd1253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:30.710419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e472d-2420-4536-a566-6ea525bd1253 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:30.723273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 984/1955] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:30.865775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14785a21-b970-4145-926b-3a94830a0e5d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:30.879788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14785a21-b970-4145-926b-3a94830a0e5d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:30.884468 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b3eab-19a2-4134-89ec-aaa63c0b289c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:30.885521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b3eab-19a2-4134-89ec-aaa63c0b289c service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:30.885910 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b3eab-19a2-4134-89ec-aaa63c0b289c service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:30.886406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b3eab-19a2-4134-89ec-aaa63c0b289c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:30.956590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b3eab-19a2-4134-89ec-aaa63c0b289c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:30.958100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b3eab-19a2-4134-89ec-aaa63c0b289c service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:30.968821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 972/1956] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:31.124357 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e472d-2420-4536-a566-6ea525bd1253 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:31.131630 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4f2e472d-2420-4536-a566-6ea525bd1253 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:31.244899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e472d-2420-4536-a566-6ea525bd1253 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:31.250267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:31.251006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:31.251653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:31.251849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] PATH_INFO: `/v3/users/f03964cc3de642bfaa9ee5a4f05bfccc` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:31.253323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03964cc3de642bfaa9ee5a4f05bfccc)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:31.255359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:31.473372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03964cc3de642bfaa9ee5a4f05bfccc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:31.474113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03964cc3de642bfaa9ee5a4f05bfccc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:31.474763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03964cc3de642bfaa9ee5a4f05bfccc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:31.483051 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03964cc3de642bfaa9ee5a4f05bfccc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:31.496073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03964cc3de642bfaa9ee5a4f05bfccc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:31.612031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 985/1957] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:31 2022] DELETE /identity/v3/users/f03964cc3de642bfaa9ee5a4f05bfccc => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:31.617085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b407e9-5194-4c83-9495-7deeb1e29eb0 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:31.617566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b407e9-5194-4c83-9495-7deeb1e29eb0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:31.617948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b407e9-5194-4c83-9495-7deeb1e29eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:31.646415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-16b407e9-5194-4c83-9495-7deeb1e29eb0 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:31.647789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-16b407e9-5194-4c83-9495-7deeb1e29eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:31.723326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 973/1958] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:31.743153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:31.744809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:31.750311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:31.805363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df0397-facf-4921-880f-7d9474a0cda6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:31.809317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca235a17-f2e5-4e4f-8fc4-6a8c1ac17b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:31.809930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca235a17-f2e5-4e4f-8fc4-6a8c1ac17b58 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:31.810211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca235a17-f2e5-4e4f-8fc4-6a8c1ac17b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:31.810484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca235a17-f2e5-4e4f-8fc4-6a8c1ac17b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:31.819292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca235a17-f2e5-4e4f-8fc4-6a8c1ac17b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:31.820748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca235a17-f2e5-4e4f-8fc4-6a8c1ac17b58 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:31.832237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 986/1959] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:32.317952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b407e9-5194-4c83-9495-7deeb1e29eb0 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:32.323751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-16b407e9-5194-4c83-9495-7deeb1e29eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:32.333927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b407e9-5194-4c83-9495-7deeb1e29eb0 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:32.352964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:32.352964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:32.352964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:32.352964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] PATH_INFO: `/v3/projects/fcd6521d51e8449b9d29a54ab3baf6c5` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:32.354392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd6521d51e8449b9d29a54ab3baf6c5)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:32.356365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:32.492535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:32.510750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd6521d51e8449b9d29a54ab3baf6c5', 'request_id': 'req-86ec5d27-7427-4001-930f-2f9bddfbf505'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:32.515792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd6521d51e8449b9d29a54ab3baf6c5', 'request_id': 'req-86ec5d27-7427-4001-930f-2f9bddfbf505'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:32.521790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd6521d51e8449b9d29a54ab3baf6c5', 'request_id': 'req-86ec5d27-7427-4001-930f-2f9bddfbf505'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:32.634355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] The token cache is being invalidate because project fcd6521d51e8449b9d29a54ab3baf6c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:32.636055 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:32.637097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-86ec5d27-7427-4001-930f-2f9bddfbf505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:32.639200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 974/1960] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:32 2022] DELETE /identity/v3/projects/fcd6521d51e8449b9d29a54ab3baf6c5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:32.645304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b7503-6247-478a-acb2-f80f33547b77 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:32.645795 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b7503-6247-478a-acb2-f80f33547b77 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:32.645871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b7503-6247-478a-acb2-f80f33547b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:32.669889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-257b7503-6247-478a-acb2-f80f33547b77 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:32.671254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-257b7503-6247-478a-acb2-f80f33547b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:32.736472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 987/1961] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:32.746474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cbc9a-6bd9-459d-bf53-fbd2349b557d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:32.746758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cbc9a-6bd9-459d-bf53-fbd2349b557d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:32.747035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cbc9a-6bd9-459d-bf53-fbd2349b557d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:32.770314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a22cbc9a-6bd9-459d-bf53-fbd2349b557d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:32.772230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a22cbc9a-6bd9-459d-bf53-fbd2349b557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:32.781024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 975/1962] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:32.785455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b7503-6247-478a-acb2-f80f33547b77 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:32.796790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b7503-6247-478a-acb2-f80f33547b77 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:32.801600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce78f-b2c3-4f85-a352-306877deacd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:32.802390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce78f-b2c3-4f85-a352-306877deacd2 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:32.802740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce78f-b2c3-4f85-a352-306877deacd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:32.803251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce78f-b2c3-4f85-a352-306877deacd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:32.803899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce78f-b2c3-4f85-a352-306877deacd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:32.804207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce78f-b2c3-4f85-a352-306877deacd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:32.805830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce78f-b2c3-4f85-a352-306877deacd2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:32.819292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 988/1963] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:52:32.823915 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cbc9a-6bd9-459d-bf53-fbd2349b557d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:32.835382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cbc9a-6bd9-459d-bf53-fbd2349b557d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:32.842895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7e8cf-1898-40b5-91b2-30891a42e797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:32.843902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7e8cf-1898-40b5-91b2-30891a42e797 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:32.844331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7e8cf-1898-40b5-91b2-30891a42e797 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:32.844797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7e8cf-1898-40b5-91b2-30891a42e797 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:32.845651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7e8cf-1898-40b5-91b2-30891a42e797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:32.847711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7e8cf-1898-40b5-91b2-30891a42e797 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:32.974359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 976/1964] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:32 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:32.976993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce78f-b2c3-4f85-a352-306877deacd2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:32.998666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce78f-b2c3-4f85-a352-306877deacd2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:33.066110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e5cf6-b209-4c54-a2b0-beb3b5043930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:33.066754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e5cf6-b209-4c54-a2b0-beb3b5043930 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:33.067033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e5cf6-b209-4c54-a2b0-beb3b5043930 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:33.067306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e5cf6-b209-4c54-a2b0-beb3b5043930 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:33.067977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e5cf6-b209-4c54-a2b0-beb3b5043930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:33.069984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e5cf6-b209-4c54-a2b0-beb3b5043930 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:33.187797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7e8cf-1898-40b5-91b2-30891a42e797 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:33.193923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 989/1965] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:32 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:33.200187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7e8cf-1898-40b5-91b2-30891a42e797 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:33.205104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de5a46-55b5-4bff-ab0b-0576f33da8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:33.205826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de5a46-55b5-4bff-ab0b-0576f33da8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:33.206114 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de5a46-55b5-4bff-ab0b-0576f33da8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:33.206453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de5a46-55b5-4bff-ab0b-0576f33da8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:33.207115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de5a46-55b5-4bff-ab0b-0576f33da8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:33.208816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de5a46-55b5-4bff-ab0b-0576f33da8f3 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:33.219876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 977/1966] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:33.225991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e5cf6-b209-4c54-a2b0-beb3b5043930 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:33.247603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e5cf6-b209-4c54-a2b0-beb3b5043930 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:33.252428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452c3e8-2c0b-4543-9bd3-a1ea66094635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:33.253061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452c3e8-2c0b-4543-9bd3-a1ea66094635 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:33.253351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452c3e8-2c0b-4543-9bd3-a1ea66094635 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:33.253625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452c3e8-2c0b-4543-9bd3-a1ea66094635 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:33.254269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452c3e8-2c0b-4543-9bd3-a1ea66094635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:33.256319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452c3e8-2c0b-4543-9bd3-a1ea66094635 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:33.270117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 990/1967] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:33.274292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de5a46-55b5-4bff-ab0b-0576f33da8f3 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:33.289969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de5a46-55b5-4bff-ab0b-0576f33da8f3 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:33.290431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994393b-5d1e-4ff7-abb5-218b494e9b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:33.292227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994393b-5d1e-4ff7-abb5-218b494e9b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:33.292227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994393b-5d1e-4ff7-abb5-218b494e9b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:33.292227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994393b-5d1e-4ff7-abb5-218b494e9b7d admin admin] PATH_INFO: `/v3/projects/0833a02fb67046aeb7f70a317e9a2e73/users/db0a62db116947df80b484dde76eab60/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:33.334090 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994393b-5d1e-4ff7-abb5-218b494e9b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0833a02fb67046aeb7f70a317e9a2e73, user_id=db0a62db116947df80b484dde76eab60, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:33.337190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994393b-5d1e-4ff7-abb5-218b494e9b7d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:33.451576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 978/1968] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:33 2022] PUT /identity/v3/projects/0833a02fb67046aeb7f70a317e9a2e73/users/db0a62db116947df80b484dde76eab60/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:33.456924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6f67f-8e64-4799-b96c-dba3bf8f4ff2 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:33.457456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6f67f-8e64-4799-b96c-dba3bf8f4ff2 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:33.457800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6f67f-8e64-4799-b96c-dba3bf8f4ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:33.487397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-82e6f67f-8e64-4799-b96c-dba3bf8f4ff2 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:33.492219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-82e6f67f-8e64-4799-b96c-dba3bf8f4ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:33.500339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:33.574343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 991/1969] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:33.584254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994393b-5d1e-4ff7-abb5-218b494e9b7d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:33.586150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2994393b-5d1e-4ff7-abb5-218b494e9b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:33.683577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994393b-5d1e-4ff7-abb5-218b494e9b7d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:33.689768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1cda1-55f0-464e-93fa-ad43f80397aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:33.690585 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1cda1-55f0-464e-93fa-ad43f80397aa service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:33.691015 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1cda1-55f0-464e-93fa-ad43f80397aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:33.691487 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1cda1-55f0-464e-93fa-ad43f80397aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:33.706386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1cda1-55f0-464e-93fa-ad43f80397aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:33.708446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1cda1-55f0-464e-93fa-ad43f80397aa service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:33.726929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 979/1970] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:34.987633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6f67f-8e64-4799-b96c-dba3bf8f4ff2 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:34.989227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-82e6f67f-8e64-4799-b96c-dba3bf8f4ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:35.002945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6f67f-8e64-4799-b96c-dba3bf8f4ff2 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:35.010384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:35.011530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:35.012464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:35.012918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] PATH_INFO: `/v3/users/6ce3d74110524ea1b9d851659efbff6e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:35.015185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ce3d74110524ea1b9d851659efbff6e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:35.018404 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:35.236642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ce3d74110524ea1b9d851659efbff6e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:35.237364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ce3d74110524ea1b9d851659efbff6e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:35.237988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ce3d74110524ea1b9d851659efbff6e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:35.255594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ce3d74110524ea1b9d851659efbff6e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:35.274454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ce3d74110524ea1b9d851659efbff6e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:35.395344 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 992/1971] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:34 2022] DELETE /identity/v3/users/6ce3d74110524ea1b9d851659efbff6e => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:35.399316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359890d1-e480-42ca-b8c4-14f49da10cfc None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:35.400483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359890d1-e480-42ca-b8c4-14f49da10cfc None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:35.400994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359890d1-e480-42ca-b8c4-14f49da10cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:35.428280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-359890d1-e480-42ca-b8c4-14f49da10cfc None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:35.431123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-359890d1-e480-42ca-b8c4-14f49da10cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:35.503477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:35.514984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 980/1972] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:35.528496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:35.530581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:35.599810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b7b8e-6ceb-483d-9760-14965d54c79c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:35.610936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e5b0f-8458-4da3-8869-2bbb2bca59ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:35.611584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e5b0f-8458-4da3-8869-2bbb2bca59ab service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:35.611864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e5b0f-8458-4da3-8869-2bbb2bca59ab service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:35.612139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e5b0f-8458-4da3-8869-2bbb2bca59ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:35.633155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e5b0f-8458-4da3-8869-2bbb2bca59ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:35.634742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e5b0f-8458-4da3-8869-2bbb2bca59ab service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:35.648933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 993/1973] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:35.962357 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359890d1-e480-42ca-b8c4-14f49da10cfc None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:35.964944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-359890d1-e480-42ca-b8c4-14f49da10cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:36.058985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359890d1-e480-42ca-b8c4-14f49da10cfc None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:36.063497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db20ba5-be49-497a-a4a1-b93ff7040be1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:36.064118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db20ba5-be49-497a-a4a1-b93ff7040be1 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:36.064401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db20ba5-be49-497a-a4a1-b93ff7040be1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:36.064677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db20ba5-be49-497a-a4a1-b93ff7040be1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:36.066725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1db20ba5-be49-497a-a4a1-b93ff7040be1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:36.172629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db20ba5-be49-497a-a4a1-b93ff7040be1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:36.176331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db20ba5-be49-497a-a4a1-b93ff7040be1 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:36.191438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 981/1974] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:36.296426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e5b0f-8458-4da3-8869-2bbb2bca59ab service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:36.298263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-903e5b0f-8458-4da3-8869-2bbb2bca59ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:36.322812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e5b0f-8458-4da3-8869-2bbb2bca59ab service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:36.336972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38225683-663b-4315-b390-951a36a1234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:36.337877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38225683-663b-4315-b390-951a36a1234e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:36.338156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38225683-663b-4315-b390-951a36a1234e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:36.338427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38225683-663b-4315-b390-951a36a1234e admin admin] PATH_INFO: `/v3/users/f0964faec07c487da00b184294d818d3` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:36.341444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38225683-663b-4315-b390-951a36a1234e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0964faec07c487da00b184294d818d3)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:36.343384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38225683-663b-4315-b390-951a36a1234e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:36.385477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:36.584135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-38225683-663b-4315-b390-951a36a1234e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0964faec07c487da00b184294d818d3'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:36.585035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-38225683-663b-4315-b390-951a36a1234e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0964faec07c487da00b184294d818d3'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:36.585770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-38225683-663b-4315-b390-951a36a1234e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0964faec07c487da00b184294d818d3'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:36.603388 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-38225683-663b-4315-b390-951a36a1234e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0964faec07c487da00b184294d818d3'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:36.619287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-38225683-663b-4315-b390-951a36a1234e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0964faec07c487da00b184294d818d3'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:36.678415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db20ba5-be49-497a-a4a1-b93ff7040be1 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:36.680611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1db20ba5-be49-497a-a4a1-b93ff7040be1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:36.698301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 994/1975] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:36 2022] DELETE /identity/v3/users/f0964faec07c487da00b184294d818d3 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:36.704090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7f09-5ae4-47aa-9233-39c001eb4ea1 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:36.705235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7f09-5ae4-47aa-9233-39c001eb4ea1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:36.705630 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7f09-5ae4-47aa-9233-39c001eb4ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:36.710385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db20ba5-be49-497a-a4a1-b93ff7040be1 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:36.737104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-9eaf7f09-5ae4-47aa-9233-39c001eb4ea1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:36.738668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9eaf7f09-5ae4-47aa-9233-39c001eb4ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:36.796936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:36.797756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:36.798165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:36.798450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] PATH_INFO: `/v3/projects/4e6f8fe12a6e4784bc03bcd7710c8b95` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:36.800281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6f8fe12a6e4784bc03bcd7710c8b95)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:36.803637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:36.868954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 995/1976] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:36.878461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf7f09-5ae4-47aa-9233-39c001eb4ea1 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:36.883195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9eaf7f09-5ae4-47aa-9233-39c001eb4ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:36.948441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf7f09-5ae4-47aa-9233-39c001eb4ea1 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:36.952721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c20a7-9608-42b9-a8c8-7e6e9e665157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:36.953522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c20a7-9608-42b9-a8c8-7e6e9e665157 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:36.953814 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c20a7-9608-42b9-a8c8-7e6e9e665157 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:36.954092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c20a7-9608-42b9-a8c8-7e6e9e665157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:36.983733 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6f8fe12a6e4784bc03bcd7710c8b95', 'request_id': 'req-b4116ddb-1d5a-46af-a97c-2df7b50c244d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:36.988297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:36.995969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6f8fe12a6e4784bc03bcd7710c8b95', 'request_id': 'req-b4116ddb-1d5a-46af-a97c-2df7b50c244d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:37.004089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6f8fe12a6e4784bc03bcd7710c8b95', 'request_id': 'req-b4116ddb-1d5a-46af-a97c-2df7b50c244d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:37.005602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c20a7-9608-42b9-a8c8-7e6e9e665157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:37.007934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c20a7-9608-42b9-a8c8-7e6e9e665157 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:37.029833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 996/1977] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:52:37.116161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] The token cache is being invalidate because project 4e6f8fe12a6e4784bc03bcd7710c8b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:37.116981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:37.119746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:37.125241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 982/1978] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:36 2022] DELETE /identity/v3/projects/4e6f8fe12a6e4784bc03bcd7710c8b95 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:37.126280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c20a7-9608-42b9-a8c8-7e6e9e665157 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:37.128948 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-bd4c20a7-9608-42b9-a8c8-7e6e9e665157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:37.147193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c20a7-9608-42b9-a8c8-7e6e9e665157 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:37.218687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:37.219394 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:37.221408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:37.225570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] PATH_INFO: `/v3/users/1933a294609b49039dc023af4b3065de` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:37.227314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] RBAC: Authorizing `identity:delete_user(user_id=1933a294609b49039dc023af4b3065de)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:37.229368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:37.611658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1933a294609b49039dc023af4b3065de'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:37.612983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1933a294609b49039dc023af4b3065de'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:37.614191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1933a294609b49039dc023af4b3065de'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:37.646054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1933a294609b49039dc023af4b3065de'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:37.670843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1933a294609b49039dc023af4b3065de'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:37.759812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 997/1979] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:37 2022] DELETE /identity/v3/users/1933a294609b49039dc023af4b3065de => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:37.829319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:37.830808 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:37.851481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4116ddb-1d5a-46af-a97c-2df7b50c244d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:37.969713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:37.969713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:37.969713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:37.969713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] PATH_INFO: `/v3/projects/f6cfe6743951447daa85f166c8cd5644` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:37.970040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6cfe6743951447daa85f166c8cd5644)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:37.971500 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:38.077275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:38.145509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6cfe6743951447daa85f166c8cd5644', 'request_id': 'req-07efb31b-57d4-4d92-8dcd-bd2807c5b142'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.164567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6cfe6743951447daa85f166c8cd5644', 'request_id': 'req-07efb31b-57d4-4d92-8dcd-bd2807c5b142'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.179882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6cfe6743951447daa85f166c8cd5644', 'request_id': 'req-07efb31b-57d4-4d92-8dcd-bd2807c5b142'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.290590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] The token cache is being invalidate because project f6cfe6743951447daa85f166c8cd5644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:38.291148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.294534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.296601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 983/1980] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:37 2022] DELETE /identity/v3/projects/f6cfe6743951447daa85f166c8cd5644 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:38.302257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:38.304216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:38.327352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebe714-c206-4906-ab9e-2c8ce757cead admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:38.506568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:38.508061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:38.516536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:38.516536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] PATH_INFO: `/v3/users/bb5ac9d0ce97480bbf0d7d85e9bae65f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:38.516536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb5ac9d0ce97480bbf0d7d85e9bae65f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:38.516536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:38.566820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:38.568594 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:38.611300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efb31b-57d4-4d92-8dcd-bd2807c5b142 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:38.619679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:38.621632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:38.622248 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:38.622764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] PATH_INFO: `/v3/projects/a6488bc18d1441c7bc824eefc4ee7462` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:38.624429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6488bc18d1441c7bc824eefc4ee7462)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:38.627233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:38.809523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6488bc18d1441c7bc824eefc4ee7462', 'request_id': 'req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.816530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb5ac9d0ce97480bbf0d7d85e9bae65f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.817717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb5ac9d0ce97480bbf0d7d85e9bae65f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.820365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6488bc18d1441c7bc824eefc4ee7462', 'request_id': 'req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.821552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb5ac9d0ce97480bbf0d7d85e9bae65f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.828581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6488bc18d1441c7bc824eefc4ee7462', 'request_id': 'req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.832580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:38.837496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb5ac9d0ce97480bbf0d7d85e9bae65f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.856291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9930ac30-731b-494d-a2be-e889345f2447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb5ac9d0ce97480bbf0d7d85e9bae65f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.918190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] The token cache is being invalidate because project a6488bc18d1441c7bc824eefc4ee7462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:38.918190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.918876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d0bdc947-471a-4c8f-ae0b-d27cfe235aa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:38.920985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 984/1981] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:38 2022] DELETE /identity/v3/projects/a6488bc18d1441c7bc824eefc4ee7462 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:38.929569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba0edc-e088-494d-9e99-e606e391278d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:38.929569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba0edc-e088-494d-9e99-e606e391278d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:38.929569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba0edc-e088-494d-9e99-e606e391278d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:39.001162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 998/1982] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:38 2022] DELETE /identity/v3/users/bb5ac9d0ce97480bbf0d7d85e9bae65f => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:39.012604 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-79ba0edc-e088-494d-9e99-e606e391278d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:39.014070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-79ba0edc-e088-494d-9e99-e606e391278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:39.166098 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 985/1983] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:39.174661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8e2b7-3093-4ca2-a0d2-a57a48ab1141 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:39.175469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8e2b7-3093-4ca2-a0d2-a57a48ab1141 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:39.176125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8e2b7-3093-4ca2-a0d2-a57a48ab1141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:39.227012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e0a8e2b7-3093-4ca2-a0d2-a57a48ab1141 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:39.230727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e0a8e2b7-3093-4ca2-a0d2-a57a48ab1141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:39.247104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 999/1984] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:39.247823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba0edc-e088-494d-9e99-e606e391278d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:39.258636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba0edc-e088-494d-9e99-e606e391278d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:39.267137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c2a20-0a28-4d3a-a2ad-24dd1940fa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:39.272561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c2a20-0a28-4d3a-a2ad-24dd1940fa54 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:39.272561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c2a20-0a28-4d3a-a2ad-24dd1940fa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:39.272561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c2a20-0a28-4d3a-a2ad-24dd1940fa54 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:39.272561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c2a20-0a28-4d3a-a2ad-24dd1940fa54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:39.272561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c2a20-0a28-4d3a-a2ad-24dd1940fa54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:39.272561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c2a20-0a28-4d3a-a2ad-24dd1940fa54 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:39.289897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 986/1985] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:52:39.294363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8e2b7-3093-4ca2-a0d2-a57a48ab1141 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:39.305403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8e2b7-3093-4ca2-a0d2-a57a48ab1141 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:39.310349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df20dfc-e846-4430-ac59-868f2ec11e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:39.310981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df20dfc-e846-4430-ac59-868f2ec11e28 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:39.311267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df20dfc-e846-4430-ac59-868f2ec11e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:39.311540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df20dfc-e846-4430-ac59-868f2ec11e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:39.312408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df20dfc-e846-4430-ac59-868f2ec11e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:39.314415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df20dfc-e846-4430-ac59-868f2ec11e28 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:39.465525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1000/1986] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:39 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:39.469959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c2a20-0a28-4d3a-a2ad-24dd1940fa54 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:39.484471 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c2a20-0a28-4d3a-a2ad-24dd1940fa54 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:39.606124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467764b-8cae-4314-9421-9bc9ba1991e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:39.607545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467764b-8cae-4314-9421-9bc9ba1991e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:39.608030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467764b-8cae-4314-9421-9bc9ba1991e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:39.608500 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467764b-8cae-4314-9421-9bc9ba1991e8 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:39.610027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467764b-8cae-4314-9421-9bc9ba1991e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:39.613248 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467764b-8cae-4314-9421-9bc9ba1991e8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:39.658995 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df20dfc-e846-4430-ac59-868f2ec11e28 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:39.660808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4df20dfc-e846-4430-ac59-868f2ec11e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:39.693885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df20dfc-e846-4430-ac59-868f2ec11e28 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:39.710273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 987/1987] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:39 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:39.711300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467764b-8cae-4314-9421-9bc9ba1991e8 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:39.712377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:39.716659 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:39.717111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:39.717579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] PATH_INFO: `/v3/projects/01b9121bdb0848d9b764e026941ce67a` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:39.719016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b9121bdb0848d9b764e026941ce67a)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:39.721549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:39.722051 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467764b-8cae-4314-9421-9bc9ba1991e8 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:39.731756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfa8e1-0c6f-4b6f-b2de-fe7022dbc65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:39.732065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:39.732386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfa8e1-0c6f-4b6f-b2de-fe7022dbc65f admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:39.732670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfa8e1-0c6f-4b6f-b2de-fe7022dbc65f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:39.732950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfa8e1-0c6f-4b6f-b2de-fe7022dbc65f admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:39.736183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfa8e1-0c6f-4b6f-b2de-fe7022dbc65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:39.740055 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfa8e1-0c6f-4b6f-b2de-fe7022dbc65f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:39.757859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 988/1988] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:39.759720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfa8e1-0c6f-4b6f-b2de-fe7022dbc65f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:39.785912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfa8e1-0c6f-4b6f-b2de-fe7022dbc65f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:39.793005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c7b9d-245b-466c-8a8f-58b482650de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:39.793823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c7b9d-245b-466c-8a8f-58b482650de8 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:39.794111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c7b9d-245b-466c-8a8f-58b482650de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:39.794388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c7b9d-245b-466c-8a8f-58b482650de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:39.795685 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c7b9d-245b-466c-8a8f-58b482650de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:39.797536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c7b9d-245b-466c-8a8f-58b482650de8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:39.815360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 989/1989] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:39.819090 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c7b9d-245b-466c-8a8f-58b482650de8 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:39.830962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c7b9d-245b-466c-8a8f-58b482650de8 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:39.836122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33125709-d9a3-4597-bf97-9ddc544f94a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:39.837898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125709-d9a3-4597-bf97-9ddc544f94a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:39.838325 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125709-d9a3-4597-bf97-9ddc544f94a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:39.838760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125709-d9a3-4597-bf97-9ddc544f94a5 admin admin] PATH_INFO: `/v3/projects/135f8764fafd4a39bee1570486a1d92f/users/cc1586d8d1e54235ab4e54daa3c6c249/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:39.861245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33125709-d9a3-4597-bf97-9ddc544f94a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=135f8764fafd4a39bee1570486a1d92f, user_id=cc1586d8d1e54235ab4e54daa3c6c249, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:39.863980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33125709-d9a3-4597-bf97-9ddc544f94a5 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:39.980504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 990/1990] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:39 2022] PUT /identity/v3/projects/135f8764fafd4a39bee1570486a1d92f/users/cc1586d8d1e54235ab4e54daa3c6c249/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:39.993750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c359b4-dfa8-4ae5-a2ac-0c6745aee104 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:39.994215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c359b4-dfa8-4ae5-a2ac-0c6745aee104 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:39.994755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c359b4-dfa8-4ae5-a2ac-0c6745aee104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:40.004049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b9121bdb0848d9b764e026941ce67a', 'request_id': 'req-e7ea5090-90a6-497e-b1c0-8139fdfea4df'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:40.017744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b9121bdb0848d9b764e026941ce67a', 'request_id': 'req-e7ea5090-90a6-497e-b1c0-8139fdfea4df'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:40.059449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-76c359b4-dfa8-4ae5-a2ac-0c6745aee104 None None] MFA Rules not processed for user `cc1586d8d1e54235ab4e54daa3c6c249`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:40.061082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-76c359b4-dfa8-4ae5-a2ac-0c6745aee104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:40.125021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b9121bdb0848d9b764e026941ce67a', 'request_id': 'req-e7ea5090-90a6-497e-b1c0-8139fdfea4df'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:40.153670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] The token cache is being invalidate because project 01b9121bdb0848d9b764e026941ce67a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:40.154084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:40.154906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e7ea5090-90a6-497e-b1c0-8139fdfea4df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:40.156498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1001/1991] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:39 2022] DELETE /identity/v3/projects/01b9121bdb0848d9b764e026941ce67a => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:40.165692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f14fa-b881-4815-ae7d-1e6684808619 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:40.166045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f14fa-b881-4815-ae7d-1e6684808619 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:40.166336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f14fa-b881-4815-ae7d-1e6684808619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:40.232798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-000f14fa-b881-4815-ae7d-1e6684808619 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:40.232798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-000f14fa-b881-4815-ae7d-1e6684808619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:40.247569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 991/1992] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:40.263553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c359b4-dfa8-4ae5-a2ac-0c6745aee104 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:40.265682 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-76c359b4-dfa8-4ae5-a2ac-0c6745aee104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:40.340444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1002/1993] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:40.346586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c9d5-006f-4c5d-b385-0dd1bfad676c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:40.346750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c9d5-006f-4c5d-b385-0dd1bfad676c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:40.347101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c9d5-006f-4c5d-b385-0dd1bfad676c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:40.374587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c359b4-dfa8-4ae5-a2ac-0c6745aee104 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:40.379740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cf807-a48e-4988-a6e7-9ee8fa25fdc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:40.380973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cf807-a48e-4988-a6e7-9ee8fa25fdc5 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:40.381457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cf807-a48e-4988-a6e7-9ee8fa25fdc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:40.382915 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cf807-a48e-4988-a6e7-9ee8fa25fdc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:40.392497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cf807-a48e-4988-a6e7-9ee8fa25fdc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:40.394436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cf807-a48e-4988-a6e7-9ee8fa25fdc5 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:40.396120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f7e5c9d5-006f-4c5d-b385-0dd1bfad676c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:40.397971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f7e5c9d5-006f-4c5d-b385-0dd1bfad676c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:40.413170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 992/1994] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:40.423456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1003/1995] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:40.427495 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cf807-a48e-4988-a6e7-9ee8fa25fdc5 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:40.440319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cf807-a48e-4988-a6e7-9ee8fa25fdc5 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:40.447166 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96f3f4-adb0-4528-90bc-cda9e04735bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:40.447872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96f3f4-adb0-4528-90bc-cda9e04735bc admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:40.448402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96f3f4-adb0-4528-90bc-cda9e04735bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:40.448821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96f3f4-adb0-4528-90bc-cda9e04735bc admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:40.449604 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96f3f4-adb0-4528-90bc-cda9e04735bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:40.449987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96f3f4-adb0-4528-90bc-cda9e04735bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:40.451655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96f3f4-adb0-4528-90bc-cda9e04735bc admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:40.481095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 993/1996] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:52:40.485355 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c9d5-006f-4c5d-b385-0dd1bfad676c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:40.497944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c9d5-006f-4c5d-b385-0dd1bfad676c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:40.506905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a414-6be1-4ebc-ae5c-8dd3bebaaee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:40.507637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a414-6be1-4ebc-ae5c-8dd3bebaaee6 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:40.509871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a414-6be1-4ebc-ae5c-8dd3bebaaee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:40.510345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399a414-6be1-4ebc-ae5c-8dd3bebaaee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:40.511106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a414-6be1-4ebc-ae5c-8dd3bebaaee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:40.515325 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399a414-6be1-4ebc-ae5c-8dd3bebaaee6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:40.575634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:40.649653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1004/1997] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:40 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:40.650954 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96f3f4-adb0-4528-90bc-cda9e04735bc admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:40.669358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96f3f4-adb0-4528-90bc-cda9e04735bc admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:40.772512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22540dd-bb93-4868-b878-4d4a6377e506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:40.773554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22540dd-bb93-4868-b878-4d4a6377e506 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:40.774644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22540dd-bb93-4868-b878-4d4a6377e506 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:40.775111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22540dd-bb93-4868-b878-4d4a6377e506 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:40.776186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22540dd-bb93-4868-b878-4d4a6377e506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:40.779482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22540dd-bb93-4868-b878-4d4a6377e506 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:40.905976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a414-6be1-4ebc-ae5c-8dd3bebaaee6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:40.907520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 994/1998] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:40 2022] POST /identity/v3/users => generated 294 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:40.923454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399a414-6be1-4ebc-ae5c-8dd3bebaaee6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:40.930517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a951d11-25f7-4bf8-9de7-2867644a0aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:40.931386 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a951d11-25f7-4bf8-9de7-2867644a0aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:40.931896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a951d11-25f7-4bf8-9de7-2867644a0aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:40.932405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a951d11-25f7-4bf8-9de7-2867644a0aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:40.933921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a951d11-25f7-4bf8-9de7-2867644a0aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:40.935994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a951d11-25f7-4bf8-9de7-2867644a0aa6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:40.946819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1005/1999] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:40.952960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22540dd-bb93-4868-b878-4d4a6377e506 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:40.967676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22540dd-bb93-4868-b878-4d4a6377e506 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:40.977032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5de3e8-6d05-434d-b005-6f0c43916cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:40.978046 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5de3e8-6d05-434d-b005-6f0c43916cce admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:40.978319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5de3e8-6d05-434d-b005-6f0c43916cce admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:40.978715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5de3e8-6d05-434d-b005-6f0c43916cce admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:40.979255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5de3e8-6d05-434d-b005-6f0c43916cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:40.981858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5de3e8-6d05-434d-b005-6f0c43916cce admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:41.005760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 995/2000] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:41.012941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a951d11-25f7-4bf8-9de7-2867644a0aa6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:41.032613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a951d11-25f7-4bf8-9de7-2867644a0aa6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:41.037660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76925ae-ddc8-4dec-8c05-e019ac73d284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:41.038328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76925ae-ddc8-4dec-8c05-e019ac73d284 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:41.038613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76925ae-ddc8-4dec-8c05-e019ac73d284 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:41.038889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76925ae-ddc8-4dec-8c05-e019ac73d284 admin admin] PATH_INFO: `/v3/projects/f2f913b4248341e6b5317b54db8a51fe/users/0b9384e04100463ab7c8562ee07046ec/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:41.055991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76925ae-ddc8-4dec-8c05-e019ac73d284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f913b4248341e6b5317b54db8a51fe, user_id=0b9384e04100463ab7c8562ee07046ec, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:41.059728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76925ae-ddc8-4dec-8c05-e019ac73d284 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:41.171175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1006/2001] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:41 2022] PUT /identity/v3/projects/f2f913b4248341e6b5317b54db8a51fe/users/0b9384e04100463ab7c8562ee07046ec/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:41.182821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041498cb-8136-4de8-92fe-19903e8254dc None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:41.183720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041498cb-8136-4de8-92fe-19903e8254dc None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:41.183995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041498cb-8136-4de8-92fe-19903e8254dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:41.220467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-041498cb-8136-4de8-92fe-19903e8254dc None None] MFA Rules not processed for user `0b9384e04100463ab7c8562ee07046ec`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:41.221249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-041498cb-8136-4de8-92fe-19903e8254dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:41.358423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76925ae-ddc8-4dec-8c05-e019ac73d284 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:41.358950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 996/2002] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:41.514315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76925ae-ddc8-4dec-8c05-e019ac73d284 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:41.565873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8445e-ac29-4830-a22c-6d64fd20b1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:41.565873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8445e-ac29-4830-a22c-6d64fd20b1bd admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:41.565873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8445e-ac29-4830-a22c-6d64fd20b1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:41.565873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8445e-ac29-4830-a22c-6d64fd20b1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:41.566832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8445e-ac29-4830-a22c-6d64fd20b1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:41.566832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8445e-ac29-4830-a22c-6d64fd20b1bd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:41.594081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041498cb-8136-4de8-92fe-19903e8254dc None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:41.598129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-041498cb-8136-4de8-92fe-19903e8254dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:41.633333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041498cb-8136-4de8-92fe-19903e8254dc None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:41.633333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:41.634033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:41.634391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:41.634736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] PATH_INFO: `/v3/users/6227b1dd03864ccbb629eeb32f05d8e7` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:41.642472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6227b1dd03864ccbb629eeb32f05d8e7)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:41.644719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:41.726059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1007/2003] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:41 2022] POST /identity/v3/projects => generated 385 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:41.730417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8445e-ac29-4830-a22c-6d64fd20b1bd admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:41.813569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8445e-ac29-4830-a22c-6d64fd20b1bd admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:41.819818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27a659-326d-4986-a091-254e0dcd1eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:41.820765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27a659-326d-4986-a091-254e0dcd1eda admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:41.821270 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27a659-326d-4986-a091-254e0dcd1eda admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:41.821710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27a659-326d-4986-a091-254e0dcd1eda admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:41.822370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27a659-326d-4986-a091-254e0dcd1eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:41.825305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27a659-326d-4986-a091-254e0dcd1eda admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:41.936503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6227b1dd03864ccbb629eeb32f05d8e7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:41.937233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6227b1dd03864ccbb629eeb32f05d8e7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:41.937901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6227b1dd03864ccbb629eeb32f05d8e7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:41.988989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1008/2004] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:41 2022] POST /identity/v3/users => generated 299 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:52:41.991904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27a659-326d-4986-a091-254e0dcd1eda admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:41.995402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3d27a659-326d-4986-a091-254e0dcd1eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:42.011248 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6227b1dd03864ccbb629eeb32f05d8e7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:42.023784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27a659-326d-4986-a091-254e0dcd1eda admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:42.031449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a1304-9dc4-4df7-9cea-649791a84c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:42.032235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a1304-9dc4-4df7-9cea-649791a84c79 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.032597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a1304-9dc4-4df7-9cea-649791a84c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.032959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a1304-9dc4-4df7-9cea-649791a84c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.033705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f0722f4a-1668-4899-91e2-4ae5f6986adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6227b1dd03864ccbb629eeb32f05d8e7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:42.035778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a1304-9dc4-4df7-9cea-649791a84c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:42.036258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a1304-9dc4-4df7-9cea-649791a84c79 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:42.050312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1009/2005] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:42.057438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a1304-9dc4-4df7-9cea-649791a84c79 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:42.076728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a1304-9dc4-4df7-9cea-649791a84c79 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:42.087146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01110a-ebe2-44af-a938-6da82289eb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:42.088007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01110a-ebe2-44af-a938-6da82289eb5b admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.088547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01110a-ebe2-44af-a938-6da82289eb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.088885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01110a-ebe2-44af-a938-6da82289eb5b admin admin] PATH_INFO: `/v3/projects/16b2e4aa091e422ba0a438554384912d/users/e77a6ab5e03c477b99f96a65a438bf17/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.112077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01110a-ebe2-44af-a938-6da82289eb5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b2e4aa091e422ba0a438554384912d, user_id=e77a6ab5e03c477b99f96a65a438bf17, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:42.115248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01110a-ebe2-44af-a938-6da82289eb5b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:42.149920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 997/2006] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:41 2022] DELETE /identity/v3/users/6227b1dd03864ccbb629eeb32f05d8e7 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:42.151503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27c24-27a1-4707-b77f-7f2a0991ddb0 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.153322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27c24-27a1-4707-b77f-7f2a0991ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.153869 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f27c24-27a1-4707-b77f-7f2a0991ddb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.201748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1010/2007] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:42 2022] PUT /identity/v3/projects/16b2e4aa091e422ba0a438554384912d/users/e77a6ab5e03c477b99f96a65a438bf17/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:42.203127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13189fe-fd85-4e29-9fdc-640ae6d14285 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.203490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13189fe-fd85-4e29-9fdc-640ae6d14285 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.203841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13189fe-fd85-4e29-9fdc-640ae6d14285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.219376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-32f27c24-27a1-4707-b77f-7f2a0991ddb0 None None] MFA Rules not processed for user `db0a62db116947df80b484dde76eab60`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:42.220770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-32f27c24-27a1-4707-b77f-7f2a0991ddb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:42.240863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a13189fe-fd85-4e29-9fdc-640ae6d14285 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:42.248531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a13189fe-fd85-4e29-9fdc-640ae6d14285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:42.311838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 998/2008] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:42.311838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f27c24-27a1-4707-b77f-7f2a0991ddb0 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:42.346537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1011/2009] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:42.346537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13189fe-fd85-4e29-9fdc-640ae6d14285 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:42.346537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a13189fe-fd85-4e29-9fdc-640ae6d14285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:42.377338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13189fe-fd85-4e29-9fdc-640ae6d14285 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:42.378549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ef307-5fbb-4607-b214-c4a041ae5ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:42.379264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ef307-5fbb-4607-b214-c4a041ae5ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.379584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ef307-5fbb-4607-b214-c4a041ae5ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.379858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ef307-5fbb-4607-b214-c4a041ae5ae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.380672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ef307-5fbb-4607-b214-c4a041ae5ae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:42.384790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ef307-5fbb-4607-b214-c4a041ae5ae6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:42.402425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f27c24-27a1-4707-b77f-7f2a0991ddb0 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:42.409173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198b30c-25af-49df-8171-a52e40c9f2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:42.410065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198b30c-25af-49df-8171-a52e40c9f2db admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.410351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198b30c-25af-49df-8171-a52e40c9f2db admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.410627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198b30c-25af-49df-8171-a52e40c9f2db admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.411281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198b30c-25af-49df-8171-a52e40c9f2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:42.412952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198b30c-25af-49df-8171-a52e40c9f2db admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:42.425748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 999/2010] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:42.426762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198b30c-25af-49df-8171-a52e40c9f2db admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:42.428470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4198b30c-25af-49df-8171-a52e40c9f2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:42.488804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198b30c-25af-49df-8171-a52e40c9f2db admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:42.492496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780dfab-1e07-480f-80c8-3c3e2d4ed810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:42.493125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780dfab-1e07-480f-80c8-3c3e2d4ed810 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.493405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780dfab-1e07-480f-80c8-3c3e2d4ed810 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.493686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780dfab-1e07-480f-80c8-3c3e2d4ed810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.508088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1012/2011] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:42 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:42.508297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780dfab-1e07-480f-80c8-3c3e2d4ed810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:42.508403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ef307-5fbb-4607-b214-c4a041ae5ae6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:42.508514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780dfab-1e07-480f-80c8-3c3e2d4ed810 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:42.517327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1000/2012] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:42.518235 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780dfab-1e07-480f-80c8-3c3e2d4ed810 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:42.528173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780dfab-1e07-480f-80c8-3c3e2d4ed810 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:42.614634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54952d4-6cfb-4fee-904f-878c564a1e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:42.617429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54952d4-6cfb-4fee-904f-878c564a1e28 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.618040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54952d4-6cfb-4fee-904f-878c564a1e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.618489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54952d4-6cfb-4fee-904f-878c564a1e28 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.619282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54952d4-6cfb-4fee-904f-878c564a1e28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:42.633290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54952d4-6cfb-4fee-904f-878c564a1e28 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:42.658463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ef307-5fbb-4607-b214-c4a041ae5ae6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:42.665583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6a879-4c2f-414e-85b7-4bd8a1758640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:42.668518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6a879-4c2f-414e-85b7-4bd8a1758640 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.668806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6a879-4c2f-414e-85b7-4bd8a1758640 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.669320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6a879-4c2f-414e-85b7-4bd8a1758640 admin admin] PATH_INFO: `/v3/projects/16b2e4aa091e422ba0a438554384912d/users/e77a6ab5e03c477b99f96a65a438bf17/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.671571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6a879-4c2f-414e-85b7-4bd8a1758640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b2e4aa091e422ba0a438554384912d, user_id=e77a6ab5e03c477b99f96a65a438bf17, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:42.680529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6a879-4c2f-414e-85b7-4bd8a1758640 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:42.690038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1013/2013] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:42 2022] PUT /identity/v3/projects/16b2e4aa091e422ba0a438554384912d/users/e77a6ab5e03c477b99f96a65a438bf17/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:42.701380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31dcd8-a510-4f2b-b0af-e1abdf03794c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.701665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31dcd8-a510-4f2b-b0af-e1abdf03794c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.702217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31dcd8-a510-4f2b-b0af-e1abdf03794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.731752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4e31dcd8-a510-4f2b-b0af-e1abdf03794c None None] MFA Rules not processed for user `e77a6ab5e03c477b99f96a65a438bf17`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:42.733412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4e31dcd8-a510-4f2b-b0af-e1abdf03794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:42.767247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1001/2014] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:42 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:42.771714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54952d4-6cfb-4fee-904f-878c564a1e28 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:42.790391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54952d4-6cfb-4fee-904f-878c564a1e28 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:42.853947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:42.864997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1014/2015] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:42.879061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31dcd8-a510-4f2b-b0af-e1abdf03794c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:42.883727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4e31dcd8-a510-4f2b-b0af-e1abdf03794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:42.910707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ab860-0ca0-46da-baa8-4e612ff84c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:42.912680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab860-0ca0-46da-baa8-4e612ff84c67 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.913252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab860-0ca0-46da-baa8-4e612ff84c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.914291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab860-0ca0-46da-baa8-4e612ff84c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.915303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ab860-0ca0-46da-baa8-4e612ff84c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:42.917724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ab860-0ca0-46da-baa8-4e612ff84c67 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:42.937433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1002/2016] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:42.939988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ab860-0ca0-46da-baa8-4e612ff84c67 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:42.955438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ab860-0ca0-46da-baa8-4e612ff84c67 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:42.960516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0964b7-700f-4f86-8f66-e3d34fca1c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:42.961220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0964b7-700f-4f86-8f66-e3d34fca1c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.961645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0964b7-700f-4f86-8f66-e3d34fca1c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.961927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0964b7-700f-4f86-8f66-e3d34fca1c54 admin admin] PATH_INFO: `/v3/projects/993dd93a784c403085f8ced9b07e9a91/users/5fb981e9468749f08821c3c45863cabc/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.966516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31dcd8-a510-4f2b-b0af-e1abdf03794c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:42.971089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77ae4d-a0fd-411b-8479-474b8909a9d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:42.971275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77ae4d-a0fd-411b-8479-474b8909a9d4 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:42.977699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77ae4d-a0fd-411b-8479-474b8909a9d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:42.977699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77ae4d-a0fd-411b-8479-474b8909a9d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:42.982430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77ae4d-a0fd-411b-8479-474b8909a9d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:42.983868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77ae4d-a0fd-411b-8479-474b8909a9d4 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:42.986914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0964b7-700f-4f86-8f66-e3d34fca1c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993dd93a784c403085f8ced9b07e9a91, user_id=5fb981e9468749f08821c3c45863cabc, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:42.990156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0964b7-700f-4f86-8f66-e3d34fca1c54 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:42.997489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1015/2017] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:43.033168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77ae4d-a0fd-411b-8479-474b8909a9d4 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:43.046593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77ae4d-a0fd-411b-8479-474b8909a9d4 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:43.051520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:43.052284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:43.052627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:43.052993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] PATH_INFO: `/v3/users/0b9384e04100463ab7c8562ee07046ec` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:43.054275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9384e04100463ab7c8562ee07046ec)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:43.056379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:43.065375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1003/2018] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:42 2022] PUT /identity/v3/projects/993dd93a784c403085f8ced9b07e9a91/users/5fb981e9468749f08821c3c45863cabc/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:43.070744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0964b7-700f-4f86-8f66-e3d34fca1c54 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:43.151445 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0964b7-700f-4f86-8f66-e3d34fca1c54 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:43.153776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f09e4-35a8-4fbc-be4b-4f3b87ee4693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:43.154504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f09e4-35a8-4fbc-be4b-4f3b87ee4693 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:43.154885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f09e4-35a8-4fbc-be4b-4f3b87ee4693 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:43.155249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f09e4-35a8-4fbc-be4b-4f3b87ee4693 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:43.156083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f09e4-35a8-4fbc-be4b-4f3b87ee4693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:43.157786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f09e4-35a8-4fbc-be4b-4f3b87ee4693 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:43.169804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1004/2019] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:43.174118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f09e4-35a8-4fbc-be4b-4f3b87ee4693 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:43.174118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-535f09e4-35a8-4fbc-be4b-4f3b87ee4693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:43.191543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f09e4-35a8-4fbc-be4b-4f3b87ee4693 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:43.197126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:43.197949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:43.198238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:43.199035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] PATH_INFO: `/v3/users/e164588320914d6695404512f37c4aa3` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:43.201033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e164588320914d6695404512f37c4aa3)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:43.203012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:43.299556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9384e04100463ab7c8562ee07046ec'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.300666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9384e04100463ab7c8562ee07046ec'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.304532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9384e04100463ab7c8562ee07046ec'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.309911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9384e04100463ab7c8562ee07046ec'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.335084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9384e04100463ab7c8562ee07046ec'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.526888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1016/2020] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:43 2022] DELETE /identity/v3/users/0b9384e04100463ab7c8562ee07046ec => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:43.527292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:43.529722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:43.531452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:43.568064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e164588320914d6695404512f37c4aa3'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.568064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e164588320914d6695404512f37c4aa3'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.568064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e164588320914d6695404512f37c4aa3'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.578609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e164588320914d6695404512f37c4aa3'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.585984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e164588320914d6695404512f37c4aa3'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.615008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfe766-b3b1-463f-af3f-4984ffbe4246 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:43.620061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:43.621273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:43.628122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:43.628122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] PATH_INFO: `/v3/projects/e083988200c74ff5af3c543013549cd3` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:43.628122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e083988200c74ff5af3c543013549cd3)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:43.628122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:43.680398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1005/2021] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:43 2022] DELETE /identity/v3/users/e164588320914d6695404512f37c4aa3 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:43.681491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:43.685507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:43.753279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533cba5-8d28-4423-a851-951552088300 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:43.753519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e083988200c74ff5af3c543013549cd3', 'request_id': 'req-0cc0affb-a69f-4a22-9da5-0659842bea6e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.782299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e083988200c74ff5af3c543013549cd3', 'request_id': 'req-0cc0affb-a69f-4a22-9da5-0659842bea6e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.832752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f6faa-3385-4759-b2a7-0fd280e711ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:43.832752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f6faa-3385-4759-b2a7-0fd280e711ef admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:43.832752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f6faa-3385-4759-b2a7-0fd280e711ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:43.832752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f6faa-3385-4759-b2a7-0fd280e711ef admin admin] PATH_INFO: `/v3/projects/993dd93a784c403085f8ced9b07e9a91/users/5fb981e9468749f08821c3c45863cabc/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:43.833467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f6faa-3385-4759-b2a7-0fd280e711ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=993dd93a784c403085f8ced9b07e9a91, user_id=5fb981e9468749f08821c3c45863cabc, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:43.836479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f6faa-3385-4759-b2a7-0fd280e711ef admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:43.847741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1006/2022] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:43 2022] PUT /identity/v3/projects/993dd93a784c403085f8ced9b07e9a91/users/5fb981e9468749f08821c3c45863cabc/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:43.853821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e083988200c74ff5af3c543013549cd3', 'request_id': 'req-0cc0affb-a69f-4a22-9da5-0659842bea6e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.855475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e22f13-d55c-44d0-80c7-2098280cdbb3 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:43.859518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e22f13-d55c-44d0-80c7-2098280cdbb3 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:43.859925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e22f13-d55c-44d0-80c7-2098280cdbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:43.889526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] The token cache is being invalidate because project e083988200c74ff5af3c543013549cd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:43.890333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.893417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cc0affb-a69f-4a22-9da5-0659842bea6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:43.897961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1017/2023] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:43 2022] DELETE /identity/v3/projects/e083988200c74ff5af3c543013549cd3 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:43.897961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc584-a840-4399-95bb-a920338d4809 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:43.897961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc584-a840-4399-95bb-a920338d4809 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:43.897961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc584-a840-4399-95bb-a920338d4809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:43.916832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-78e22f13-d55c-44d0-80c7-2098280cdbb3 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:43.920975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-78e22f13-d55c-44d0-80c7-2098280cdbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:43.940941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-0afdc584-a840-4399-95bb-a920338d4809 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:43.942813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0afdc584-a840-4399-95bb-a920338d4809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:44.007235 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1007/2024] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:44.008536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e22f13-d55c-44d0-80c7-2098280cdbb3 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:44.011795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-78e22f13-d55c-44d0-80c7-2098280cdbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:44.040659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e22f13-d55c-44d0-80c7-2098280cdbb3 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:44.040659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:44.041294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:44.041578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:44.041924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] PATH_INFO: `/v3/users/e6a8a7865b5b43358ce54f944e14e9ea` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:44.043280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6a8a7865b5b43358ce54f944e14e9ea)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:44.045247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:44.055666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1018/2025] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:44.056825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdc584-a840-4399-95bb-a920338d4809 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:44.061027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0afdc584-a840-4399-95bb-a920338d4809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:44.159702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdc584-a840-4399-95bb-a920338d4809 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:44.173315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc9446-ab4f-435a-8290-63970b6d5114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:44.174887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc9446-ab4f-435a-8290-63970b6d5114 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:44.175524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc9446-ab4f-435a-8290-63970b6d5114 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:44.175978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc9446-ab4f-435a-8290-63970b6d5114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:44.200574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc9446-ab4f-435a-8290-63970b6d5114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:44.202305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc9446-ab4f-435a-8290-63970b6d5114 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:44.216231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1019/2026] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:44.222134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc9446-ab4f-435a-8290-63970b6d5114 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:44.233222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc9446-ab4f-435a-8290-63970b6d5114 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:44.237649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa83584-a41a-41aa-a1cb-9acad6cbcc80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:44.238951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa83584-a41a-41aa-a1cb-9acad6cbcc80 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:44.239252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa83584-a41a-41aa-a1cb-9acad6cbcc80 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:44.239533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa83584-a41a-41aa-a1cb-9acad6cbcc80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:44.251120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa83584-a41a-41aa-a1cb-9acad6cbcc80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:44.252563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa83584-a41a-41aa-a1cb-9acad6cbcc80 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:44.267618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1020/2027] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:44.274410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6a8a7865b5b43358ce54f944e14e9ea'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:44.275100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6a8a7865b5b43358ce54f944e14e9ea'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:44.275860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6a8a7865b5b43358ce54f944e14e9ea'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:44.305816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6a8a7865b5b43358ce54f944e14e9ea'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:44.326481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6a8a7865b5b43358ce54f944e14e9ea'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:44.405652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1008/2028] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:44 2022] DELETE /identity/v3/users/e6a8a7865b5b43358ce54f944e14e9ea => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:44.940763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:44.999421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:45.163340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa83584-a41a-41aa-a1cb-9acad6cbcc80 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:45.165963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3fa83584-a41a-41aa-a1cb-9acad6cbcc80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:45.185692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa83584-a41a-41aa-a1cb-9acad6cbcc80 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:45.366930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:45.367613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:45.367906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:45.368181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] PATH_INFO: `/v3/projects/f2f913b4248341e6b5317b54db8a51fe` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:45.369586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f913b4248341e6b5317b54db8a51fe)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:45.372679 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:45.440075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:45.441705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:45.459883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c45c9-72ba-4ba5-a45b-593db2ce73c8 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:45.468118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:45.469264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:45.469771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:45.470263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] PATH_INFO: `/v3/projects/663f572aa9ff4073900815457db49595` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:45.471734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] RBAC: Authorizing `identity:delete_project(project_id=663f572aa9ff4073900815457db49595)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:45.474435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:45.633461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f913b4248341e6b5317b54db8a51fe', 'request_id': 'req-69d1a08d-8ecd-41b2-a46f-01a39813cd91'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:45.648735 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f913b4248341e6b5317b54db8a51fe', 'request_id': 'req-69d1a08d-8ecd-41b2-a46f-01a39813cd91'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:45.681844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '663f572aa9ff4073900815457db49595', 'request_id': 'req-66c7b5a5-3a34-4306-8368-04ba1f44234c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:45.699215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '663f572aa9ff4073900815457db49595', 'request_id': 'req-66c7b5a5-3a34-4306-8368-04ba1f44234c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:45.712813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '663f572aa9ff4073900815457db49595', 'request_id': 'req-66c7b5a5-3a34-4306-8368-04ba1f44234c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:45.782631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f913b4248341e6b5317b54db8a51fe', 'request_id': 'req-69d1a08d-8ecd-41b2-a46f-01a39813cd91'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:45.796402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] The token cache is being invalidate because project f2f913b4248341e6b5317b54db8a51fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:45.796857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:45.797574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:45.799365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1021/2029] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:45 2022] DELETE /identity/v3/projects/f2f913b4248341e6b5317b54db8a51fe => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:45.800049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:45.801406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:45.883797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1a08d-8ecd-41b2-a46f-01a39813cd91 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:45.911763 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] The token cache is being invalidate because project 663f572aa9ff4073900815457db49595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:45.917154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:45.917154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:45.952410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1009/2030] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:45 2022] DELETE /identity/v3/projects/663f572aa9ff4073900815457db49595 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:45.957306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:45.959255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:45.973180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:45.989430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:45.989430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:45.989430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] PATH_INFO: `/v3/projects/688fabe7d978462f89bc5be042ef1128` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:45.991674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=688fabe7d978462f89bc5be042ef1128)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:45.997934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:46.006823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b5a5-3a34-4306-8368-04ba1f44234c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:46.048795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:46.050289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:46.052339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:46.053384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] PATH_INFO: `/v3/users/e77a6ab5e03c477b99f96a65a438bf17` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:46.070727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77a6ab5e03c477b99f96a65a438bf17)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:46.072982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:46.391021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688fabe7d978462f89bc5be042ef1128', 'request_id': 'req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:46.398507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77a6ab5e03c477b99f96a65a438bf17'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:46.399545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77a6ab5e03c477b99f96a65a438bf17'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:46.399794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688fabe7d978462f89bc5be042ef1128', 'request_id': 'req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:46.400695 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77a6ab5e03c477b99f96a65a438bf17'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:46.425170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77a6ab5e03c477b99f96a65a438bf17'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:46.437116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77a6ab5e03c477b99f96a65a438bf17'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:46.474491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688fabe7d978462f89bc5be042ef1128', 'request_id': 'req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:46.494291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] The token cache is being invalidate because project 688fabe7d978462f89bc5be042ef1128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:46.494697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:46.496034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e4f6a0b4-b8e0-4519-b834-a13da1c3ec2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:46.501143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1022/2031] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:45 2022] DELETE /identity/v3/projects/688fabe7d978462f89bc5be042ef1128 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:46.501143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309c09f-f83f-4600-84f4-4848610e1335 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:46.501143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309c09f-f83f-4600-84f4-4848610e1335 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:46.501143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309c09f-f83f-4600-84f4-4848610e1335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:46.544112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d309c09f-f83f-4600-84f4-4848610e1335 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:46.557102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d309c09f-f83f-4600-84f4-4848610e1335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:46.577267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1010/2032] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:45 2022] DELETE /identity/v3/users/e77a6ab5e03c477b99f96a65a438bf17 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:46.578823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:46.584305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:46.674980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:46.719143 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520842af-e34d-420d-bb83-5465e7907825 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:46.732302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:46.732975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:46.740912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1023/2033] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:46.741290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:46.741290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] PATH_INFO: `/v3/users/37a51029dff141ed9ffa05d7c70d87d6` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:46.741290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a51029dff141ed9ffa05d7c70d87d6)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:46.741290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:46.742609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c5162-eef1-4dad-a12d-5ddd2ff53ceb None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:46.742968 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c5162-eef1-4dad-a12d-5ddd2ff53ceb None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:46.743690 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c5162-eef1-4dad-a12d-5ddd2ff53ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:46.774018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-cf4c5162-eef1-4dad-a12d-5ddd2ff53ceb None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:46.775929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-cf4c5162-eef1-4dad-a12d-5ddd2ff53ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:46.804745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1024/2034] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:46.809672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c5162-eef1-4dad-a12d-5ddd2ff53ceb None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:46.831513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c5162-eef1-4dad-a12d-5ddd2ff53ceb None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:46.841084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32f4ce-e4aa-42a4-8995-416a176a18a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:46.841957 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32f4ce-e4aa-42a4-8995-416a176a18a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:46.842266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32f4ce-e4aa-42a4-8995-416a176a18a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:46.842529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32f4ce-e4aa-42a4-8995-416a176a18a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:46.843268 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32f4ce-e4aa-42a4-8995-416a176a18a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:46.843556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32f4ce-e4aa-42a4-8995-416a176a18a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:46.845174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32f4ce-e4aa-42a4-8995-416a176a18a0 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:46.866494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1025/2035] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:52:46.873806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32f4ce-e4aa-42a4-8995-416a176a18a0 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:46.884801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32f4ce-e4aa-42a4-8995-416a176a18a0 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:46.892114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f20058-e63f-40b2-8a12-b1ab6cdbf7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:46.893515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f20058-e63f-40b2-8a12-b1ab6cdbf7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:46.893876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f20058-e63f-40b2-8a12-b1ab6cdbf7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:46.894367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f20058-e63f-40b2-8a12-b1ab6cdbf7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:46.895394 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f20058-e63f-40b2-8a12-b1ab6cdbf7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:46.897954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f20058-e63f-40b2-8a12-b1ab6cdbf7a3 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:47.010189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a51029dff141ed9ffa05d7c70d87d6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:47.011562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a51029dff141ed9ffa05d7c70d87d6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:47.012549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a51029dff141ed9ffa05d7c70d87d6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:47.020524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1026/2036] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:46 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:47.027038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f20058-e63f-40b2-8a12-b1ab6cdbf7a3 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:47.029548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-31f20058-e63f-40b2-8a12-b1ab6cdbf7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:47.048358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a51029dff141ed9ffa05d7c70d87d6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:47.066522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a51029dff141ed9ffa05d7c70d87d6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:47.138912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f20058-e63f-40b2-8a12-b1ab6cdbf7a3 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:47.143919 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144851e-56d0-43bc-ad87-da2dfd9f4a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:47.144535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144851e-56d0-43bc-ad87-da2dfd9f4a05 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:47.144942 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144851e-56d0-43bc-ad87-da2dfd9f4a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:47.145263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144851e-56d0-43bc-ad87-da2dfd9f4a05 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:47.146072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144851e-56d0-43bc-ad87-da2dfd9f4a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:47.147943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144851e-56d0-43bc-ad87-da2dfd9f4a05 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:47.189445 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1011/2037] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:46 2022] DELETE /identity/v3/users/37a51029dff141ed9ffa05d7c70d87d6 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:47.272819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1027/2038] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:47 2022] POST /identity/v3/users => generated 284 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:47.274788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:47.380394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3d959-894e-49f1-9cc6-a31a4cbdf10b admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:47.385562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182e778-6a9c-458c-959d-9e0693816ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:47.386262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182e778-6a9c-458c-959d-9e0693816ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:47.386621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182e778-6a9c-458c-959d-9e0693816ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:47.387082 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182e778-6a9c-458c-959d-9e0693816ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:47.387828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182e778-6a9c-458c-959d-9e0693816ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:47.389559 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182e778-6a9c-458c-959d-9e0693816ce8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:47.400234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1012/2039] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:47.405582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144851e-56d0-43bc-ad87-da2dfd9f4a05 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:47.419013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144851e-56d0-43bc-ad87-da2dfd9f4a05 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:47.423980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e00e37-2396-44db-8d8d-17e7c302de86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:47.424606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e00e37-2396-44db-8d8d-17e7c302de86 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:47.425119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e00e37-2396-44db-8d8d-17e7c302de86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:47.425907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e00e37-2396-44db-8d8d-17e7c302de86 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:47.426760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e00e37-2396-44db-8d8d-17e7c302de86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:47.428477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e00e37-2396-44db-8d8d-17e7c302de86 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:47.470326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1028/2040] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:47.475928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182e778-6a9c-458c-959d-9e0693816ce8 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:47.492865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e00e37-2396-44db-8d8d-17e7c302de86 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:47.496281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-03e00e37-2396-44db-8d8d-17e7c302de86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:47.503333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182e778-6a9c-458c-959d-9e0693816ce8 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:47.514711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cf691-6278-4459-bb61-c5c9e9ec1b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:47.515644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cf691-6278-4459-bb61-c5c9e9ec1b77 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:47.515934 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cf691-6278-4459-bb61-c5c9e9ec1b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:47.516217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cf691-6278-4459-bb61-c5c9e9ec1b77 admin admin] PATH_INFO: `/v3/projects/fc7dff044bcf426f88fc3f559e5077da/users/fe8d6c497246448db70aa9ce1fff71c6/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:47.520618 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e00e37-2396-44db-8d8d-17e7c302de86 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:47.530007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:47.532214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:47.532821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:47.533305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] PATH_INFO: `/v3/projects/16b2e4aa091e422ba0a438554384912d` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:47.534700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b2e4aa091e422ba0a438554384912d)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:47.536916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:47.560378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cf691-6278-4459-bb61-c5c9e9ec1b77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7dff044bcf426f88fc3f559e5077da, user_id=fe8d6c497246448db70aa9ce1fff71c6, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:47.564536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cf691-6278-4459-bb61-c5c9e9ec1b77 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:47.685409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1013/2041] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:47 2022] PUT /identity/v3/projects/fc7dff044bcf426f88fc3f559e5077da/users/fe8d6c497246448db70aa9ce1fff71c6/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:47.695882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b638c94-f22d-441c-81cf-248e8a286b55 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:47.696364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b638c94-f22d-441c-81cf-248e8a286b55 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:47.696640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b638c94-f22d-441c-81cf-248e8a286b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:47.697613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b2e4aa091e422ba0a438554384912d', 'request_id': 'req-d2906573-a58b-43eb-9110-b5426bf47001'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:47.705334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b2e4aa091e422ba0a438554384912d', 'request_id': 'req-d2906573-a58b-43eb-9110-b5426bf47001'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:47.715424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:47.729154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-0b638c94-f22d-441c-81cf-248e8a286b55 None None] MFA Rules not processed for user `fe8d6c497246448db70aa9ce1fff71c6`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:47.730738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0b638c94-f22d-441c-81cf-248e8a286b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:47.770606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b2e4aa091e422ba0a438554384912d', 'request_id': 'req-d2906573-a58b-43eb-9110-b5426bf47001'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:47.784079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] The token cache is being invalidate because project 16b2e4aa091e422ba0a438554384912d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:47.784582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:47.785444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d2906573-a58b-43eb-9110-b5426bf47001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:47.790455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1029/2042] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:47 2022] DELETE /identity/v3/projects/16b2e4aa091e422ba0a438554384912d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:47.794413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f3540-d212-48e8-a985-b0cda4d9281f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:47.794817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f3540-d212-48e8-a985-b0cda4d9281f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:47.795291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f3540-d212-48e8-a985-b0cda4d9281f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:47.809278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1014/2043] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:47.818563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b638c94-f22d-441c-81cf-248e8a286b55 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:47.818563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0b638c94-f22d-441c-81cf-248e8a286b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:47.850549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-727f3540-d212-48e8-a985-b0cda4d9281f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:47.852939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-727f3540-d212-48e8-a985-b0cda4d9281f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:47.958901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b638c94-f22d-441c-81cf-248e8a286b55 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:47.973796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62867853-08db-4b31-9969-deb3a91321f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:47.976747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867853-08db-4b31-9969-deb3a91321f0 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:47.977057 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867853-08db-4b31-9969-deb3a91321f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:47.978491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867853-08db-4b31-9969-deb3a91321f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:47.979977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62867853-08db-4b31-9969-deb3a91321f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:47.988524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1030/2044] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:47.990552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62867853-08db-4b31-9969-deb3a91321f0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:47.991236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0c521-57db-4256-9478-d3adfcf9c17f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:47.991531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0c521-57db-4256-9478-d3adfcf9c17f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:47.991810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0c521-57db-4256-9478-d3adfcf9c17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:48.034375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a7d0c521-57db-4256-9478-d3adfcf9c17f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:48.038131 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a7d0c521-57db-4256-9478-d3adfcf9c17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:48.054661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1031/2045] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:48.066509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0c521-57db-4256-9478-d3adfcf9c17f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:48.093894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0c521-57db-4256-9478-d3adfcf9c17f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:48.122751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d3650-5b8c-482b-9268-a68c21fea82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:48.123408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d3650-5b8c-482b-9268-a68c21fea82c admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:48.123695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d3650-5b8c-482b-9268-a68c21fea82c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:48.124003 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d3650-5b8c-482b-9268-a68c21fea82c admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:48.124642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d3650-5b8c-482b-9268-a68c21fea82c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:48.124962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d3650-5b8c-482b-9268-a68c21fea82c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:48.126739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d3650-5b8c-482b-9268-a68c21fea82c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:48.156598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1032/2046] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:52:48.157732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d3650-5b8c-482b-9268-a68c21fea82c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:48.170034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d3650-5b8c-482b-9268-a68c21fea82c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:48.176358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4c8a1-48e0-4566-aae8-6c471adc9c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:48.177207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4c8a1-48e0-4566-aae8-6c471adc9c99 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:48.177566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4c8a1-48e0-4566-aae8-6c471adc9c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:48.179116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4c8a1-48e0-4566-aae8-6c471adc9c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:48.179478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4c8a1-48e0-4566-aae8-6c471adc9c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:48.181671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4c8a1-48e0-4566-aae8-6c471adc9c99 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:48.271090 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1015/2047] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:47 2022] POST /identity/v3/projects => generated 367 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:48.279601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62867853-08db-4b31-9969-deb3a91321f0 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:48.297037 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-62867853-08db-4b31-9969-deb3a91321f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:48.369913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1033/2048] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:48 2022] POST /identity/v3/projects => generated 361 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:48.372613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:48.372613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4c8a1-48e0-4566-aae8-6c471adc9c99 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:48.461148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62867853-08db-4b31-9969-deb3a91321f0 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:48.553496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4c8a1-48e0-4566-aae8-6c471adc9c99 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:48.555546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c89ec1-3d2e-4858-af7e-40f320ce5de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:48.556352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c89ec1-3d2e-4858-af7e-40f320ce5de0 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:48.556856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c89ec1-3d2e-4858-af7e-40f320ce5de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:48.557345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c89ec1-3d2e-4858-af7e-40f320ce5de0 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:48.558318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c89ec1-3d2e-4858-af7e-40f320ce5de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:48.560649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c89ec1-3d2e-4858-af7e-40f320ce5de0 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:48.654003 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1034/2049] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:48 2022] POST /identity/v3/users => generated 290 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:48.656188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:48.670005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c89ec1-3d2e-4858-af7e-40f320ce5de0 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:48.674052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:48.674052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:48.674052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] PATH_INFO: `/v3/projects/c8e5f2e677d44ee1a4a31a4118d93d37` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:48.679935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e5f2e677d44ee1a4a31a4118d93d37)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:48.683545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:48.711907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c89ec1-3d2e-4858-af7e-40f320ce5de0 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:48.730018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5da56a-5399-4379-ad9d-a953f4d6419d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:48.736336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5da56a-5399-4379-ad9d-a953f4d6419d admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:48.736336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5da56a-5399-4379-ad9d-a953f4d6419d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:48.736336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5da56a-5399-4379-ad9d-a953f4d6419d admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:48.736336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5da56a-5399-4379-ad9d-a953f4d6419d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:48.740658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5da56a-5399-4379-ad9d-a953f4d6419d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:48.869917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1035/2050] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:48 2022] POST /identity/v3/users => generated 281 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:48.872204 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5da56a-5399-4379-ad9d-a953f4d6419d admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:48.893071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5da56a-5399-4379-ad9d-a953f4d6419d admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:48.900130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c232a24-0e46-40f8-8b8c-0a0deb58f72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:48.900422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c232a24-0e46-40f8-8b8c-0a0deb58f72f admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:48.901466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c232a24-0e46-40f8-8b8c-0a0deb58f72f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:48.901931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c232a24-0e46-40f8-8b8c-0a0deb58f72f admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:48.903205 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c232a24-0e46-40f8-8b8c-0a0deb58f72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:48.904666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c232a24-0e46-40f8-8b8c-0a0deb58f72f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:48.919820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e5f2e677d44ee1a4a31a4118d93d37', 'request_id': 'req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:48.920387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1036/2051] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:48.920387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c232a24-0e46-40f8-8b8c-0a0deb58f72f admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:48.933534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c232a24-0e46-40f8-8b8c-0a0deb58f72f admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:48.950697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e5f2e677d44ee1a4a31a4118d93d37', 'request_id': 'req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:48.960880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e5f2e677d44ee1a4a31a4118d93d37', 'request_id': 'req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:49.007889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed95831-bf2c-460b-99c6-60c0ad4524c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:49.008549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed95831-bf2c-460b-99c6-60c0ad4524c3 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.008835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed95831-bf2c-460b-99c6-60c0ad4524c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.009145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed95831-bf2c-460b-99c6-60c0ad4524c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.009805 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed95831-bf2c-460b-99c6-60c0ad4524c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:49.011711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed95831-bf2c-460b-99c6-60c0ad4524c3 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:49.035372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1037/2052] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:49.036248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed95831-bf2c-460b-99c6-60c0ad4524c3 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:49.049730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed95831-bf2c-460b-99c6-60c0ad4524c3 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:49.054932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608b166-5322-4747-ab0e-fb1ce46a6115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:49.058692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608b166-5322-4747-ab0e-fb1ce46a6115 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.058692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608b166-5322-4747-ab0e-fb1ce46a6115 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.058692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608b166-5322-4747-ab0e-fb1ce46a6115 admin admin] PATH_INFO: `/v3/projects/771cd5be3b53406fb99ec100aa7cfddd/users/dbf528702e5343a28920854388864e5c/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.075131 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608b166-5322-4747-ab0e-fb1ce46a6115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771cd5be3b53406fb99ec100aa7cfddd, user_id=dbf528702e5343a28920854388864e5c, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:49.084554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] The token cache is being invalidate because project c8e5f2e677d44ee1a4a31a4118d93d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:49.084850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:49.085567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:49.087386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1016/2053] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:48 2022] DELETE /identity/v3/projects/c8e5f2e677d44ee1a4a31a4118d93d37 => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:49.088191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:49.089570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608b166-5322-4747-ab0e-fb1ce46a6115 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:49.092784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:49.108521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9ad1-c12a-40b2-8cfc-f0d10d551072 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:49.119618 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e0e09-9851-4941-9773-ef703e2b8623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:49.122381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e0e09-9851-4941-9773-ef703e2b8623 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.122381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e0e09-9851-4941-9773-ef703e2b8623 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.122381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e0e09-9851-4941-9773-ef703e2b8623 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.122381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e0e09-9851-4941-9773-ef703e2b8623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:49.126830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e0e09-9851-4941-9773-ef703e2b8623 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:49.141682 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1017/2054] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:49.141682 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a86d8-c6e0-4294-8343-bf18163554b9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.141682 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a86d8-c6e0-4294-8343-bf18163554b9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.141682 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a86d8-c6e0-4294-8343-bf18163554b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.163019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-3a4a86d8-c6e0-4294-8343-bf18163554b9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:49.164289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3a4a86d8-c6e0-4294-8343-bf18163554b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:49.174247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1018/2055] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:49.175074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a86d8-c6e0-4294-8343-bf18163554b9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:49.187294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a86d8-c6e0-4294-8343-bf18163554b9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:49.191040 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1038/2056] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:49 2022] PUT /identity/v3/projects/771cd5be3b53406fb99ec100aa7cfddd/users/dbf528702e5343a28920854388864e5c/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:49.193611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa640e93-4e38-4887-a2b8-591728921bc0 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.194305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa640e93-4e38-4887-a2b8-591728921bc0 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.194305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa640e93-4e38-4887-a2b8-591728921bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.244037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-aa640e93-4e38-4887-a2b8-591728921bc0 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:49.245839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-aa640e93-4e38-4887-a2b8-591728921bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:49.271318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a6802-0c06-45db-8bff-e5c0fae35d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:49.272176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a6802-0c06-45db-8bff-e5c0fae35d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.272463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a6802-0c06-45db-8bff-e5c0fae35d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.272739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a6802-0c06-45db-8bff-e5c0fae35d47 admin admin] PATH_INFO: `/v3/projects/d4e4b9765a214378b8d5f9728bf31249/users/8cc7320369a0499cbe13377e69ee74ef/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.312307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a6802-0c06-45db-8bff-e5c0fae35d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e4b9765a214378b8d5f9728bf31249, user_id=8cc7320369a0499cbe13377e69ee74ef, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:49.315200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a6802-0c06-45db-8bff-e5c0fae35d47 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:49.338723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1039/2057] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:49.342527 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa640e93-4e38-4887-a2b8-591728921bc0 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:49.342527 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-aa640e93-4e38-4887-a2b8-591728921bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:49.355562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa640e93-4e38-4887-a2b8-591728921bc0 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:49.361483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a028d78-3b70-49fc-a6b0-9ce53dfd71d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:49.362266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a028d78-3b70-49fc-a6b0-9ce53dfd71d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.362620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a028d78-3b70-49fc-a6b0-9ce53dfd71d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.363391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a028d78-3b70-49fc-a6b0-9ce53dfd71d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.363934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a028d78-3b70-49fc-a6b0-9ce53dfd71d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:49.365621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a028d78-3b70-49fc-a6b0-9ce53dfd71d4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:49.366437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:49.377251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1040/2058] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:49.378229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a028d78-3b70-49fc-a6b0-9ce53dfd71d4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:49.407087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1019/2059] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:49 2022] PUT /identity/v3/projects/d4e4b9765a214378b8d5f9728bf31249/users/8cc7320369a0499cbe13377e69ee74ef/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:49.407087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a6802-0c06-45db-8bff-e5c0fae35d47 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:49.407375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a028d78-3b70-49fc-a6b0-9ce53dfd71d4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:49.516756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b46f58-a4ff-49a1-91be-c09fda05f93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:49.518248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b46f58-a4ff-49a1-91be-c09fda05f93c admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.518248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b46f58-a4ff-49a1-91be-c09fda05f93c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.518831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b46f58-a4ff-49a1-91be-c09fda05f93c admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.520103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b46f58-a4ff-49a1-91be-c09fda05f93c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:49.520412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b46f58-a4ff-49a1-91be-c09fda05f93c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:49.524049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b46f58-a4ff-49a1-91be-c09fda05f93c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:49.546493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1041/2060] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:52:49.548763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8de02e-3866-462b-8817-859a9a49bd39 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.549162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8de02e-3866-462b-8817-859a9a49bd39 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.549537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8de02e-3866-462b-8817-859a9a49bd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.567322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a6802-0c06-45db-8bff-e5c0fae35d47 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:49.582799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865b879-e6fb-4b26-849d-1a6c1ba19d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:49.583486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865b879-e6fb-4b26-849d-1a6c1ba19d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.583824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865b879-e6fb-4b26-849d-1a6c1ba19d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.584103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865b879-e6fb-4b26-849d-1a6c1ba19d40 admin admin] PATH_INFO: `/v3/projects/771cd5be3b53406fb99ec100aa7cfddd/users/dbf528702e5343a28920854388864e5c/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.596499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4a8de02e-3866-462b-8817-859a9a49bd39 None None] MFA Rules not processed for user `8cc7320369a0499cbe13377e69ee74ef`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:49.597507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865b879-e6fb-4b26-849d-1a6c1ba19d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771cd5be3b53406fb99ec100aa7cfddd, user_id=dbf528702e5343a28920854388864e5c, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:49.599320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4a8de02e-3866-462b-8817-859a9a49bd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:49.601302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865b879-e6fb-4b26-849d-1a6c1ba19d40 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:49.621517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1020/2061] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:49 2022] PUT /identity/v3/projects/771cd5be3b53406fb99ec100aa7cfddd/users/dbf528702e5343a28920854388864e5c/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:49.622734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865b879-e6fb-4b26-849d-1a6c1ba19d40 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:49.703499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1042/2062] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:49.704845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b404d8-4aad-4cf4-8a1e-5a9eb8ffcb88 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.705172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b404d8-4aad-4cf4-8a1e-5a9eb8ffcb88 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.705887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b404d8-4aad-4cf4-8a1e-5a9eb8ffcb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.738889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865b879-e6fb-4b26-849d-1a6c1ba19d40 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:49.748924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3b1a4-e4ab-4d7e-baa2-247231e99127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:49.749614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3b1a4-e4ab-4d7e-baa2-247231e99127 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:49.749901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3b1a4-e4ab-4d7e-baa2-247231e99127 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:49.750185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3b1a4-e4ab-4d7e-baa2-247231e99127 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:49.751072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3b1a4-e4ab-4d7e-baa2-247231e99127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:49.753338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3b1a4-e4ab-4d7e-baa2-247231e99127 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:49.780924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a4b404d8-4aad-4cf4-8a1e-5a9eb8ffcb88 None None] MFA Rules not processed for user `dbf528702e5343a28920854388864e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:49.783005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a4b404d8-4aad-4cf4-8a1e-5a9eb8ffcb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:49.865365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1021/2063] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:49 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:49.865365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3b1a4-e4ab-4d7e-baa2-247231e99127 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:49.867745 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-fad3b1a4-e4ab-4d7e-baa2-247231e99127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:49.947806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1043/2064] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:49.950360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b404d8-4aad-4cf4-8a1e-5a9eb8ffcb88 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:49.973087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b404d8-4aad-4cf4-8a1e-5a9eb8ffcb88 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:50.020376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3b1a4-e4ab-4d7e-baa2-247231e99127 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:50.026629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b011c-239a-4321-993e-05493a086fca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:50.027323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b011c-239a-4321-993e-05493a086fca service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:50.027606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b011c-239a-4321-993e-05493a086fca service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:50.027887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b011c-239a-4321-993e-05493a086fca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:50.097797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b011c-239a-4321-993e-05493a086fca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:50.100461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b011c-239a-4321-993e-05493a086fca service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:50.118507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1022/2065] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:50.137319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327afa7-e36f-4e4c-a356-5c52ef36024e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:50.137319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327afa7-e36f-4e4c-a356-5c52ef36024e admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:50.137319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327afa7-e36f-4e4c-a356-5c52ef36024e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:50.137319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327afa7-e36f-4e4c-a356-5c52ef36024e admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:50.138609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327afa7-e36f-4e4c-a356-5c52ef36024e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:50.139195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b011c-239a-4321-993e-05493a086fca service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:50.144902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327afa7-e36f-4e4c-a356-5c52ef36024e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:50.162974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b011c-239a-4321-993e-05493a086fca service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:50.177645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:50.178967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:50.178967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:50.178967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] PATH_INFO: `/v3/users/8cc7320369a0499cbe13377e69ee74ef` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:50.181001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc7320369a0499cbe13377e69ee74ef)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:50.183622 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:50.262642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1044/2066] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:49 2022] POST /identity/v3/users => generated 289 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:50.267592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327afa7-e36f-4e4c-a356-5c52ef36024e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:50.288884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327afa7-e36f-4e4c-a356-5c52ef36024e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:50.288884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffc147-65ff-4822-8a89-6440fbc0b108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:50.288884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffc147-65ff-4822-8a89-6440fbc0b108 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:50.288884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffc147-65ff-4822-8a89-6440fbc0b108 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:50.290130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffc147-65ff-4822-8a89-6440fbc0b108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:50.374327 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffc147-65ff-4822-8a89-6440fbc0b108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:50.375852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffc147-65ff-4822-8a89-6440fbc0b108 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:50.386978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1045/2067] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:50.387753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffc147-65ff-4822-8a89-6440fbc0b108 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:50.398318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffc147-65ff-4822-8a89-6440fbc0b108 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:50.403390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce8399-b13c-4b75-a5fb-7d993774bed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:50.404000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8399-b13c-4b75-a5fb-7d993774bed2 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:50.404333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8399-b13c-4b75-a5fb-7d993774bed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:50.404612 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce8399-b13c-4b75-a5fb-7d993774bed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:50.405281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce8399-b13c-4b75-a5fb-7d993774bed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:50.406873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce8399-b13c-4b75-a5fb-7d993774bed2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:50.417391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1046/2068] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:50.421790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce8399-b13c-4b75-a5fb-7d993774bed2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:50.432025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce8399-b13c-4b75-a5fb-7d993774bed2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:50.437467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49505f0-80dc-4cd4-846f-db5d4ceae3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:50.438073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49505f0-80dc-4cd4-846f-db5d4ceae3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:50.438359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49505f0-80dc-4cd4-846f-db5d4ceae3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:50.438637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49505f0-80dc-4cd4-846f-db5d4ceae3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:50.439278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49505f0-80dc-4cd4-846f-db5d4ceae3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:50.440916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49505f0-80dc-4cd4-846f-db5d4ceae3d6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:50.447594 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc7320369a0499cbe13377e69ee74ef'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:50.448552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc7320369a0499cbe13377e69ee74ef'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:50.449704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc7320369a0499cbe13377e69ee74ef'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:50.452045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1047/2069] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:52:50.456720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:50.462619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49505f0-80dc-4cd4-846f-db5d4ceae3d6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:50.468246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c49505f0-80dc-4cd4-846f-db5d4ceae3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:50.495962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc7320369a0499cbe13377e69ee74ef'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:50.499385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49505f0-80dc-4cd4-846f-db5d4ceae3d6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:50.507458 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc7320369a0499cbe13377e69ee74ef'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:50.509915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecea2c-9004-4ced-a2d4-97e1c811fceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:50.510701 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecea2c-9004-4ced-a2d4-97e1c811fceb admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:50.512376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecea2c-9004-4ced-a2d4-97e1c811fceb admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:50.512561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecea2c-9004-4ced-a2d4-97e1c811fceb admin admin] PATH_INFO: `/v3/projects/0c9ea7220b974db79488cbe3ec7a4b39/users/16c9ec1a00414f9db636b13c169af63b/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:50.547210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecea2c-9004-4ced-a2d4-97e1c811fceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9ea7220b974db79488cbe3ec7a4b39, user_id=16c9ec1a00414f9db636b13c169af63b, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:50.550285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ecea2c-9004-4ced-a2d4-97e1c811fceb admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:50.663389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1023/2070] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:50 2022] DELETE /identity/v3/users/8cc7320369a0499cbe13377e69ee74ef => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:50.664633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:50.669427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1048/2071] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:50 2022] PUT /identity/v3/projects/0c9ea7220b974db79488cbe3ec7a4b39/users/16c9ec1a00414f9db636b13c169af63b/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:50.670753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:50.682150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321375b-690b-40b5-a195-3a184f6fe448 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:50.682449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321375b-690b-40b5-a195-3a184f6fe448 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:50.682873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321375b-690b-40b5-a195-3a184f6fe448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:50.739555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4321375b-690b-40b5-a195-3a184f6fe448 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:50.740877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4321375b-690b-40b5-a195-3a184f6fe448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:50.773717 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf3aa8-98d7-4421-a42f-23287b78aa64 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:50.795150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dc1b5-9be0-4f12-9ea1-1f1ac0235996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:50.796148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dc1b5-9be0-4f12-9ea1-1f1ac0235996 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:50.796666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dc1b5-9be0-4f12-9ea1-1f1ac0235996 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:50.801814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dc1b5-9be0-4f12-9ea1-1f1ac0235996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:50.801814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-884dc1b5-9be0-4f12-9ea1-1f1ac0235996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:50.910707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1049/2072] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:50.914861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc41d82-7d0f-428b-8242-fd643dcf2816 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:50.914861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc41d82-7d0f-428b-8242-fd643dcf2816 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:50.915516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc41d82-7d0f-428b-8242-fd643dcf2816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:50.939730 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884dc1b5-9be0-4f12-9ea1-1f1ac0235996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:50.939730 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884dc1b5-9be0-4f12-9ea1-1f1ac0235996 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:50.966183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1024/2073] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:50.977922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dc1b5-9be0-4f12-9ea1-1f1ac0235996 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:50.981039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-884dc1b5-9be0-4f12-9ea1-1f1ac0235996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:50.988588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8bc41d82-7d0f-428b-8242-fd643dcf2816 None None] MFA Rules not processed for user `16c9ec1a00414f9db636b13c169af63b`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:50.990238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8bc41d82-7d0f-428b-8242-fd643dcf2816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:51.132632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dc1b5-9be0-4f12-9ea1-1f1ac0235996 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:51.142627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e234aec-3bcb-410d-9706-2694b9d7ab9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:51.143499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e234aec-3bcb-410d-9706-2694b9d7ab9f service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:51.145479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e234aec-3bcb-410d-9706-2694b9d7ab9f service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:51.145774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e234aec-3bcb-410d-9706-2694b9d7ab9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:51.171681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1050/2074] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:51.172197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e234aec-3bcb-410d-9706-2694b9d7ab9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:51.172197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e234aec-3bcb-410d-9706-2694b9d7ab9f service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:51.183186 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc41d82-7d0f-428b-8242-fd643dcf2816 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:51.184870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8bc41d82-7d0f-428b-8242-fd643dcf2816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:51.197939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1025/2075] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:51.217136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:51.334221 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc41d82-7d0f-428b-8242-fd643dcf2816 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:51.351998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e22a6a-f707-4932-8aa9-a7862e51e1a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d10585480f41608bc6935e282f7e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:51.352159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e22a6a-f707-4932-8aa9-a7862e51e1a4 service glance] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:51.352401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e22a6a-f707-4932-8aa9-a7862e51e1a4 service glance] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:51.352684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e22a6a-f707-4932-8aa9-a7862e51e1a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:51.394520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e22a6a-f707-4932-8aa9-a7862e51e1a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:51.396069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e22a6a-f707-4932-8aa9-a7862e51e1a4 service glance] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:51.406570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1051/2076] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:52:52.154164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:52.178905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e234aec-3bcb-410d-9706-2694b9d7ab9f service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:52.181943 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6e234aec-3bcb-410d-9706-2694b9d7ab9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:52.216479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e234aec-3bcb-410d-9706-2694b9d7ab9f service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:52.226916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:52.227666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:52.227950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:52.228222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] PATH_INFO: `/v3/projects/d4e4b9765a214378b8d5f9728bf31249` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:52.230540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e4b9765a214378b8d5f9728bf31249)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:52.238081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:52.451513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e4b9765a214378b8d5f9728bf31249', 'request_id': 'req-c4ee8598-e4ce-403c-9256-7148438c49de'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:52.462368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e4b9765a214378b8d5f9728bf31249', 'request_id': 'req-c4ee8598-e4ce-403c-9256-7148438c49de'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:52.471911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e4b9765a214378b8d5f9728bf31249', 'request_id': 'req-c4ee8598-e4ce-403c-9256-7148438c49de'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:52.522631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f7e1b-11c5-48d8-8b7d-e4c81337702d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:52.523048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f7e1b-11c5-48d8-8b7d-e4c81337702d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:52.523368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f7e1b-11c5-48d8-8b7d-e4c81337702d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:52.573203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] The token cache is being invalidate because project d4e4b9765a214378b8d5f9728bf31249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:52.574791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:52.576595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c4ee8598-e4ce-403c-9256-7148438c49de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:52.578760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1026/2077] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:52 2022] DELETE /identity/v3/projects/d4e4b9765a214378b8d5f9728bf31249 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:52.583428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2e5fc-2173-48cc-9cda-a78a5f2dcdae None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:52.584006 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2e5fc-2173-48cc-9cda-a78a5f2dcdae None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:52.584565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2e5fc-2173-48cc-9cda-a78a5f2dcdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:52.611405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-441f7e1b-11c5-48d8-8b7d-e4c81337702d None None] MFA Rules not processed for user `5fb981e9468749f08821c3c45863cabc`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:52.611405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-441f7e1b-11c5-48d8-8b7d-e4c81337702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:52.702214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-30a2e5fc-2173-48cc-9cda-a78a5f2dcdae None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:52.702214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-30a2e5fc-2173-48cc-9cda-a78a5f2dcdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:52.722345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1052/2078] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:52.737328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f7e1b-11c5-48d8-8b7d-e4c81337702d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:52.739444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-441f7e1b-11c5-48d8-8b7d-e4c81337702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:52.742239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1027/2079] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:52.750752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2290040-3f9a-4717-b982-9a7a223175fd None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:52.751178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2290040-3f9a-4717-b982-9a7a223175fd None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:52.751551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2290040-3f9a-4717-b982-9a7a223175fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:52.790943 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a2290040-3f9a-4717-b982-9a7a223175fd None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:52.792696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a2290040-3f9a-4717-b982-9a7a223175fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:52.802986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1028/2080] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:52.807749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2290040-3f9a-4717-b982-9a7a223175fd None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:52.818172 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2290040-3f9a-4717-b982-9a7a223175fd None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:52.822982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813ae7a-6086-4a7d-b467-859abd7dcba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:52.828266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813ae7a-6086-4a7d-b467-859abd7dcba0 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:52.828266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813ae7a-6086-4a7d-b467-859abd7dcba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:52.828266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813ae7a-6086-4a7d-b467-859abd7dcba0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:52.828266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813ae7a-6086-4a7d-b467-859abd7dcba0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:52.828266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813ae7a-6086-4a7d-b467-859abd7dcba0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:52.828266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813ae7a-6086-4a7d-b467-859abd7dcba0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:52.833085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f7e1b-11c5-48d8-8b7d-e4c81337702d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:52.839164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bc7e0-0b7a-4afe-9972-0aea2230ac43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:52.839728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bc7e0-0b7a-4afe-9972-0aea2230ac43 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:52.840190 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bc7e0-0b7a-4afe-9972-0aea2230ac43 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:52.840311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1029/2081] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:52:52.840489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bc7e0-0b7a-4afe-9972-0aea2230ac43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:52.842293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-588bc7e0-0b7a-4afe-9972-0aea2230ac43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:52.844924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813ae7a-6086-4a7d-b467-859abd7dcba0 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:52.855660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813ae7a-6086-4a7d-b467-859abd7dcba0 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:52.861035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05e180-1e6c-4e14-9a54-01d4fa2c6852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:52.861897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e180-1e6c-4e14-9a54-01d4fa2c6852 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:52.862127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e180-1e6c-4e14-9a54-01d4fa2c6852 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:52.862414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e180-1e6c-4e14-9a54-01d4fa2c6852 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:52.863090 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05e180-1e6c-4e14-9a54-01d4fa2c6852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:52.865229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05e180-1e6c-4e14-9a54-01d4fa2c6852 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:52.913677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bc7e0-0b7a-4afe-9972-0aea2230ac43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:52.915520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bc7e0-0b7a-4afe-9972-0aea2230ac43 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:52.926877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1053/2082] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:52.982622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bc7e0-0b7a-4afe-9972-0aea2230ac43 service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:52.990415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1030/2083] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:52 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:53.001359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bc7e0-0b7a-4afe-9972-0aea2230ac43 service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:53.137447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb518-323e-4782-b860-b3c32de1a4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:53.138238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb518-323e-4782-b860-b3c32de1a4bc admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:53.138603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb518-323e-4782-b860-b3c32de1a4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:53.138954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74feb518-323e-4782-b860-b3c32de1a4bc admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:53.139698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74feb518-323e-4782-b860-b3c32de1a4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:53.141804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74feb518-323e-4782-b860-b3c32de1a4bc admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:53.154270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:53.257722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1054/2084] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:52 2022] POST /identity/v3/users => generated 289 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:53.258931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05e180-1e6c-4e14-9a54-01d4fa2c6852 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:53.276862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05e180-1e6c-4e14-9a54-01d4fa2c6852 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:53.284169 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a377a-b5e7-4172-9f74-482f9e3e85ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:53.284821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a377a-b5e7-4172-9f74-482f9e3e85ee admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:53.285133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a377a-b5e7-4172-9f74-482f9e3e85ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:53.285414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a377a-b5e7-4172-9f74-482f9e3e85ee admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:53.286069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a377a-b5e7-4172-9f74-482f9e3e85ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:53.287696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a377a-b5e7-4172-9f74-482f9e3e85ee admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:53.309243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1031/2085] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:53.313280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb518-323e-4782-b860-b3c32de1a4bc admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:53.325554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74feb518-323e-4782-b860-b3c32de1a4bc admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:53.337682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa8952-f8cf-4e9a-a764-274b5ad2eec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:53.338311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa8952-f8cf-4e9a-a764-274b5ad2eec0 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:53.338645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa8952-f8cf-4e9a-a764-274b5ad2eec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:53.338899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa8952-f8cf-4e9a-a764-274b5ad2eec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:53.346002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa8952-f8cf-4e9a-a764-274b5ad2eec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:53.346002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa8952-f8cf-4e9a-a764-274b5ad2eec0 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:53.354685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1055/2086] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:53.363872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a377a-b5e7-4172-9f74-482f9e3e85ee admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:53.376161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a377a-b5e7-4172-9f74-482f9e3e85ee admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:53.383348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9ad17-fb4c-4d7b-b470-c4bde5f9a663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:53.385308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9ad17-fb4c-4d7b-b470-c4bde5f9a663 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:53.385609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9ad17-fb4c-4d7b-b470-c4bde5f9a663 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:53.387562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9ad17-fb4c-4d7b-b470-c4bde5f9a663 admin admin] PATH_INFO: `/v3/projects/1e6b50c9b12e495f89c0ba254957522d/users/60cdd6ffe8324569b78cd91d82df9e42/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:53.409304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9ad17-fb4c-4d7b-b470-c4bde5f9a663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6b50c9b12e495f89c0ba254957522d, user_id=60cdd6ffe8324569b78cd91d82df9e42, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:53.412014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9ad17-fb4c-4d7b-b470-c4bde5f9a663 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:53.515668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1032/2087] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:53 2022] PUT /identity/v3/projects/1e6b50c9b12e495f89c0ba254957522d/users/60cdd6ffe8324569b78cd91d82df9e42/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:53.525805 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1149f3-f600-4378-8302-2a5a5cdbcd63 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:53.530466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1149f3-f600-4378-8302-2a5a5cdbcd63 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:53.530466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1149f3-f600-4378-8302-2a5a5cdbcd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:53.612914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-dd1149f3-f600-4378-8302-2a5a5cdbcd63 None None] MFA Rules not processed for user `60cdd6ffe8324569b78cd91d82df9e42`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:53.618337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-dd1149f3-f600-4378-8302-2a5a5cdbcd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:53.758278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1056/2088] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:53.774857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9ad17-fb4c-4d7b-b470-c4bde5f9a663 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:53.778290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cda9ad17-fb4c-4d7b-b470-c4bde5f9a663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:53.948257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:53.953889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9ad17-fb4c-4d7b-b470-c4bde5f9a663 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:53.958913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12c079-7fe0-4179-a5b6-b5d8e1c4ce88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:53.959759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12c079-7fe0-4179-a5b6-b5d8e1c4ce88 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:53.960363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12c079-7fe0-4179-a5b6-b5d8e1c4ce88 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:53.960972 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12c079-7fe0-4179-a5b6-b5d8e1c4ce88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:53.974432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12c079-7fe0-4179-a5b6-b5d8e1c4ce88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:53.976374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12c079-7fe0-4179-a5b6-b5d8e1c4ce88 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:53.995770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1033/2089] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:55.050364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:56.291042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:56.938559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1149f3-f600-4378-8302-2a5a5cdbcd63 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:56.940914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-dd1149f3-f600-4378-8302-2a5a5cdbcd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:57.087550 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1149f3-f600-4378-8302-2a5a5cdbcd63 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:57.096470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:57.097323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:57.097596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:57.098071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] PATH_INFO: `/v3/users/58303c18a6314dbeb1994d7813671e2e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:57.102831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58303c18a6314dbeb1994d7813671e2e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:57.105638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:57.172753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16912b5-65f7-4079-94be-b9cab8ebf5b8 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:57.173338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16912b5-65f7-4079-94be-b9cab8ebf5b8 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:57.173434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16912b5-65f7-4079-94be-b9cab8ebf5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:57.194812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f16912b5-65f7-4079-94be-b9cab8ebf5b8 None None] MFA Rules not processed for user `db0a62db116947df80b484dde76eab60`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:57.196047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f16912b5-65f7-4079-94be-b9cab8ebf5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:57.283939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1034/2090] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:57.396238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58303c18a6314dbeb1994d7813671e2e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:57.396900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58303c18a6314dbeb1994d7813671e2e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:57.397569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58303c18a6314dbeb1994d7813671e2e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:57.405909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58303c18a6314dbeb1994d7813671e2e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:57.419896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58303c18a6314dbeb1994d7813671e2e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:57.541157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1057/2091] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:56 2022] DELETE /identity/v3/users/58303c18a6314dbeb1994d7813671e2e => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:57.796484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16912b5-65f7-4079-94be-b9cab8ebf5b8 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:57.820927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16912b5-65f7-4079-94be-b9cab8ebf5b8 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:57.882494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:57.883121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:57.883398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:57.883668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] PATH_INFO: `/v3/projects/abc8e52cd8ec470b9fbfadb9cf01b9f7` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:57.884830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] RBAC: Authorizing `identity:delete_project(project_id=abc8e52cd8ec470b9fbfadb9cf01b9f7)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:57.887767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:57.921806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:57.923324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:57.937181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29381a8e-0109-4232-bfc5-6d6ac3649758 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:57.944034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:57.944034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:57.944436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:57.944711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] PATH_INFO: `/v3/users/fe8d6c497246448db70aa9ce1fff71c6` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:57.946050 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe8d6c497246448db70aa9ce1fff71c6)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:57.947990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:58.033414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:58.044864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abc8e52cd8ec470b9fbfadb9cf01b9f7', 'request_id': 'req-ccc2502b-35ba-40fe-b33b-5089b70d425c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:58.052717 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abc8e52cd8ec470b9fbfadb9cf01b9f7', 'request_id': 'req-ccc2502b-35ba-40fe-b33b-5089b70d425c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:58.060405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abc8e52cd8ec470b9fbfadb9cf01b9f7', 'request_id': 'req-ccc2502b-35ba-40fe-b33b-5089b70d425c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:58.143175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] The token cache is being invalidate because project abc8e52cd8ec470b9fbfadb9cf01b9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:58.143475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:58.144315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ccc2502b-35ba-40fe-b33b-5089b70d425c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:58.157319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1035/2092] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:57 2022] DELETE /identity/v3/projects/abc8e52cd8ec470b9fbfadb9cf01b9f7 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:58.157319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cd0e4-0b05-4999-b9f4-9c540b33ffd9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:58.157319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cd0e4-0b05-4999-b9f4-9c540b33ffd9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:58.157319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cd0e4-0b05-4999-b9f4-9c540b33ffd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:58.195130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-168cd0e4-0b05-4999-b9f4-9c540b33ffd9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:58.202134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-168cd0e4-0b05-4999-b9f4-9c540b33ffd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:58.217199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe8d6c497246448db70aa9ce1fff71c6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:58.222075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe8d6c497246448db70aa9ce1fff71c6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:58.223102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe8d6c497246448db70aa9ce1fff71c6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:58.243909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe8d6c497246448db70aa9ce1fff71c6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:58.273558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4dace170-4679-40ab-b842-bfa33a7a6119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe8d6c497246448db70aa9ce1fff71c6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:58.309540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1036/2093] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:58.326670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180242e4-6d9e-4e0d-b925-8261dcf9c4d4 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:58.326670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180242e4-6d9e-4e0d-b925-8261dcf9c4d4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:58.326670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180242e4-6d9e-4e0d-b925-8261dcf9c4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:58.337259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-180242e4-6d9e-4e0d-b925-8261dcf9c4d4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:58.338553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-180242e4-6d9e-4e0d-b925-8261dcf9c4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:58.349491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1037/2094] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:58.352350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180242e4-6d9e-4e0d-b925-8261dcf9c4d4 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:58.352990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1058/2095] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:57 2022] DELETE /identity/v3/users/fe8d6c497246448db70aa9ce1fff71c6 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:52:58.358009 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46485db-ef75-4a2a-a65e-ab9acecc27cb None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:58.358508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46485db-ef75-4a2a-a65e-ab9acecc27cb None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:58.358881 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46485db-ef75-4a2a-a65e-ab9acecc27cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:58.362889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180242e4-6d9e-4e0d-b925-8261dcf9c4d4 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:58.380470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b46485db-ef75-4a2a-a65e-ab9acecc27cb None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:58.381806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b46485db-ef75-4a2a-a65e-ab9acecc27cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:58.443409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bb9cb-35c5-4506-9bba-71a2b77e3aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:58.446813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bb9cb-35c5-4506-9bba-71a2b77e3aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:58.446813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bb9cb-35c5-4506-9bba-71a2b77e3aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:58.446813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bb9cb-35c5-4506-9bba-71a2b77e3aa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:58.446813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bb9cb-35c5-4506-9bba-71a2b77e3aa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:52:58.446813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bb9cb-35c5-4506-9bba-71a2b77e3aa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:58.447385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bb9cb-35c5-4506-9bba-71a2b77e3aa9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:58.458320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1059/2096] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:52:58.458823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1038/2097] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:52:58.475091 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bb9cb-35c5-4506-9bba-71a2b77e3aa9 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:58.480769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46485db-ef75-4a2a-a65e-ab9acecc27cb None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:58.482173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b46485db-ef75-4a2a-a65e-ab9acecc27cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:58.486777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bb9cb-35c5-4506-9bba-71a2b77e3aa9 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:58.491795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdc23f-d5cb-4f23-b2d8-cec9f255adab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:58.494253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdc23f-d5cb-4f23-b2d8-cec9f255adab admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:58.494253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdc23f-d5cb-4f23-b2d8-cec9f255adab admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:58.494253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdc23f-d5cb-4f23-b2d8-cec9f255adab admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:58.494253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdc23f-d5cb-4f23-b2d8-cec9f255adab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:58.495758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdc23f-d5cb-4f23-b2d8-cec9f255adab admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:58.546582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46485db-ef75-4a2a-a65e-ab9acecc27cb None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:58.550187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a2b35-2571-436c-88cf-96ffe3bf3bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:58.550819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a2b35-2571-436c-88cf-96ffe3bf3bdd service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:58.551097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a2b35-2571-436c-88cf-96ffe3bf3bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:58.551416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a2b35-2571-436c-88cf-96ffe3bf3bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:58.560422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a2b35-2571-436c-88cf-96ffe3bf3bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:58.561874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a2b35-2571-436c-88cf-96ffe3bf3bdd service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:58.571337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1060/2098] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:52:58.637633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1039/2099] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:52:58 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:58.639158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a2b35-2571-436c-88cf-96ffe3bf3bdd service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:58.649198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a2b35-2571-436c-88cf-96ffe3bf3bdd service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:58.710017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdfe7e-64fb-4056-b63c-5ad0859d5eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:58.710621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdfe7e-64fb-4056-b63c-5ad0859d5eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:58.710899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdfe7e-64fb-4056-b63c-5ad0859d5eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:58.711172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdfe7e-64fb-4056-b63c-5ad0859d5eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:58.711811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdfe7e-64fb-4056-b63c-5ad0859d5eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:58.714391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdfe7e-64fb-4056-b63c-5ad0859d5eb8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:58.794283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1061/2100] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:52:58 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:52:58.799294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdc23f-d5cb-4f23-b2d8-cec9f255adab admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:58.810641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdc23f-d5cb-4f23-b2d8-cec9f255adab admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:58.815450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9c067-5813-4f3d-83a6-e35e4cda829e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:58.816079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9c067-5813-4f3d-83a6-e35e4cda829e admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:58.816361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9c067-5813-4f3d-83a6-e35e4cda829e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:58.816633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9c067-5813-4f3d-83a6-e35e4cda829e admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:58.817323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9c067-5813-4f3d-83a6-e35e4cda829e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:58.818967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9c067-5813-4f3d-83a6-e35e4cda829e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:58.831280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1040/2101] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:58.835519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdfe7e-64fb-4056-b63c-5ad0859d5eb8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:58.846839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdfe7e-64fb-4056-b63c-5ad0859d5eb8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:58.851943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989e178-bc9a-4278-9cc4-fd377ecdbb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:58.852552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989e178-bc9a-4278-9cc4-fd377ecdbb0d admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:58.852831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989e178-bc9a-4278-9cc4-fd377ecdbb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:58.853142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989e178-bc9a-4278-9cc4-fd377ecdbb0d admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:58.853785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989e178-bc9a-4278-9cc4-fd377ecdbb0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:58.855406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989e178-bc9a-4278-9cc4-fd377ecdbb0d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:58.868980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1062/2102] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:52:58.870186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9c067-5813-4f3d-83a6-e35e4cda829e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:58.880513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9c067-5813-4f3d-83a6-e35e4cda829e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:58.886170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f88dc4-3e31-457e-8083-6b1d476654e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:58.887752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f88dc4-3e31-457e-8083-6b1d476654e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:58.887752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f88dc4-3e31-457e-8083-6b1d476654e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:58.887752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f88dc4-3e31-457e-8083-6b1d476654e9 admin admin] PATH_INFO: `/v3/projects/8e876e59e01b4871ad98bc6d30bf396d/users/8da410d2ea0740dc8eeea87244fc1408/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:58.903089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f88dc4-3e31-457e-8083-6b1d476654e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e876e59e01b4871ad98bc6d30bf396d, user_id=8da410d2ea0740dc8eeea87244fc1408, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:58.905832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f88dc4-3e31-457e-8083-6b1d476654e9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:58.994074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1041/2103] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:52:58 2022] PUT /identity/v3/projects/8e876e59e01b4871ad98bc6d30bf396d/users/8da410d2ea0740dc8eeea87244fc1408/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:59.001107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989e178-bc9a-4278-9cc4-fd377ecdbb0d admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:59.001107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8989e178-bc9a-4278-9cc4-fd377ecdbb0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:59.003711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563120a6-398d-45dd-8831-a78a46361adb None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:59.004626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563120a6-398d-45dd-8831-a78a46361adb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:59.004626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563120a6-398d-45dd-8831-a78a46361adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:59.028276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989e178-bc9a-4278-9cc4-fd377ecdbb0d admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:59.028674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-563120a6-398d-45dd-8831-a78a46361adb None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:52:59.029180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-563120a6-398d-45dd-8831-a78a46361adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:52:59.069188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:59.083250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:59.084120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:59.084400 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:59.084672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] PATH_INFO: `/v3/projects/fc7dff044bcf426f88fc3f559e5077da` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:59.085847 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7dff044bcf426f88fc3f559e5077da)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:59.087757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:59.090561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1042/2104] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:52:59.101232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563120a6-398d-45dd-8831-a78a46361adb None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:59.173179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563120a6-398d-45dd-8831-a78a46361adb None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:59.178585 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33e545-f343-4f65-9823-336fb8cd587f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:59.179304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33e545-f343-4f65-9823-336fb8cd587f service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:59.182677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33e545-f343-4f65-9823-336fb8cd587f service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:59.182677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33e545-f343-4f65-9823-336fb8cd587f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:59.191577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33e545-f343-4f65-9823-336fb8cd587f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:59.192195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7dff044bcf426f88fc3f559e5077da', 'request_id': 'req-6f9dc870-1f7e-4aee-a129-0811bcc9788f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:59.193432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33e545-f343-4f65-9823-336fb8cd587f service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:59.199548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7dff044bcf426f88fc3f559e5077da', 'request_id': 'req-6f9dc870-1f7e-4aee-a129-0811bcc9788f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:59.205568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1043/2105] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:52:59.277955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7dff044bcf426f88fc3f559e5077da', 'request_id': 'req-6f9dc870-1f7e-4aee-a129-0811bcc9788f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:59.292254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] The token cache is being invalidate because project fc7dff044bcf426f88fc3f559e5077da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:52:59.292780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:59.293510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:59.295418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1063/2106] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:52:58 2022] DELETE /identity/v3/projects/fc7dff044bcf426f88fc3f559e5077da => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:52:59.299651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33e545-f343-4f65-9823-336fb8cd587f service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:52:59.310417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33e545-f343-4f65-9823-336fb8cd587f service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:52:59.374035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:52:59.374709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:52:59.374993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:52:59.375291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] PATH_INFO: `/v3/users/dbf528702e5343a28920854388864e5c` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:52:59.376470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf528702e5343a28920854388864e5c)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:52:59.378477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:52:59.541724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:52:59.624168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf528702e5343a28920854388864e5c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:59.624847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf528702e5343a28920854388864e5c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:59.625502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf528702e5343a28920854388864e5c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:59.637846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf528702e5343a28920854388864e5c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:59.644862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f85fe132-8b5d-43a6-a159-0e6c6d0382f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf528702e5343a28920854388864e5c'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:52:59.749858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1044/2107] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:52:59 2022] DELETE /identity/v3/users/dbf528702e5343a28920854388864e5c => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:00.197779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:00.197779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:00.210854 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dc870-1f7e-4aee-a129-0811bcc9788f admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:00.274781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:00.278577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:00.279287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:00.283745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:00.283745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] PATH_INFO: `/v3/projects/771cd5be3b53406fb99ec100aa7cfddd` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:00.283745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=771cd5be3b53406fb99ec100aa7cfddd)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:00.283745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:00.403221 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771cd5be3b53406fb99ec100aa7cfddd', 'request_id': 'req-1be9c343-053b-4b84-a623-2888e7fb01b8'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:00.410477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771cd5be3b53406fb99ec100aa7cfddd', 'request_id': 'req-1be9c343-053b-4b84-a623-2888e7fb01b8'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:00.501950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771cd5be3b53406fb99ec100aa7cfddd', 'request_id': 'req-1be9c343-053b-4b84-a623-2888e7fb01b8'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:00.521825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] The token cache is being invalidate because project 771cd5be3b53406fb99ec100aa7cfddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:00.522254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:00.523095 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-1be9c343-053b-4b84-a623-2888e7fb01b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:00.525348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1064/2108] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:00 2022] DELETE /identity/v3/projects/771cd5be3b53406fb99ec100aa7cfddd => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:00.530095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f4b16-1605-439c-822c-42e25cb823f5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:00.530597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f4b16-1605-439c-822c-42e25cb823f5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:00.530676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f4b16-1605-439c-822c-42e25cb823f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:00.555623 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c85f4b16-1605-439c-822c-42e25cb823f5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:00.557283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c85f4b16-1605-439c-822c-42e25cb823f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:00.640240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1045/2109] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:00.648217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a2f7a-7b90-4278-8280-27e02e3602bc None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:00.648698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a2f7a-7b90-4278-8280-27e02e3602bc None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:00.649179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a2f7a-7b90-4278-8280-27e02e3602bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:00.672823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-ee1a2f7a-7b90-4278-8280-27e02e3602bc None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:00.674133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ee1a2f7a-7b90-4278-8280-27e02e3602bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:00.683028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1065/2110] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:00.687460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f4b16-1605-439c-822c-42e25cb823f5 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:00.698162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f4b16-1605-439c-822c-42e25cb823f5 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:00.703757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44632ae-3167-4482-8414-82257eaf0b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:00.704691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44632ae-3167-4482-8414-82257eaf0b35 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:00.711250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44632ae-3167-4482-8414-82257eaf0b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:00.711250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44632ae-3167-4482-8414-82257eaf0b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:00.711250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44632ae-3167-4482-8414-82257eaf0b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:53:00.711250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44632ae-3167-4482-8414-82257eaf0b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:00.711250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44632ae-3167-4482-8414-82257eaf0b35 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:00.725901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1046/2111] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:53:00.732894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a2f7a-7b90-4278-8280-27e02e3602bc None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:00.754442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a2f7a-7b90-4278-8280-27e02e3602bc None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:00.761759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6791cb-acea-4b8e-90cc-d92f550b0d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:00.765232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6791cb-acea-4b8e-90cc-d92f550b0d2d admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:00.765232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6791cb-acea-4b8e-90cc-d92f550b0d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:00.765232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6791cb-acea-4b8e-90cc-d92f550b0d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:00.765232 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6791cb-acea-4b8e-90cc-d92f550b0d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:00.766885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6791cb-acea-4b8e-90cc-d92f550b0d2d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:00.900326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1066/2112] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:00 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:00.903026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44632ae-3167-4482-8414-82257eaf0b35 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:00.914685 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44632ae-3167-4482-8414-82257eaf0b35 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:00.960266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:00.998386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8903545-ad17-4c5a-bf06-1c7168aefc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:00.999617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8903545-ad17-4c5a-bf06-1c7168aefc49 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:01.000135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8903545-ad17-4c5a-bf06-1c7168aefc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:01.000590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8903545-ad17-4c5a-bf06-1c7168aefc49 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:01.001316 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8903545-ad17-4c5a-bf06-1c7168aefc49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:01.003577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8903545-ad17-4c5a-bf06-1c7168aefc49 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:01.100648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1047/2113] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:00 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:01.101950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6791cb-acea-4b8e-90cc-d92f550b0d2d admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:01.114070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6791cb-acea-4b8e-90cc-d92f550b0d2d admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:01.118703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc70b9-3a98-4100-b1bc-233fedd881b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:01.119306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc70b9-3a98-4100-b1bc-233fedd881b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:01.119599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc70b9-3a98-4100-b1bc-233fedd881b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:01.119858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc70b9-3a98-4100-b1bc-233fedd881b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:01.120575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc70b9-3a98-4100-b1bc-233fedd881b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:01.122348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc70b9-3a98-4100-b1bc-233fedd881b8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:01.133382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1067/2114] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:01.138558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8903545-ad17-4c5a-bf06-1c7168aefc49 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:01.157180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8903545-ad17-4c5a-bf06-1c7168aefc49 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:01.163107 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cca3ea-a751-4ad6-90ad-1005f4f08cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:01.163977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca3ea-a751-4ad6-90ad-1005f4f08cff admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:01.164335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca3ea-a751-4ad6-90ad-1005f4f08cff admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:01.164605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca3ea-a751-4ad6-90ad-1005f4f08cff admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:01.165496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cca3ea-a751-4ad6-90ad-1005f4f08cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:01.168492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cca3ea-a751-4ad6-90ad-1005f4f08cff admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:01.185456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1048/2115] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:53:01.194033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc70b9-3a98-4100-b1bc-233fedd881b8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:01.206331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc70b9-3a98-4100-b1bc-233fedd881b8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:01.211794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697c968-675c-47eb-940a-e087874307df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:01.212509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697c968-675c-47eb-940a-e087874307df admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:01.212790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697c968-675c-47eb-940a-e087874307df admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:01.213082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697c968-675c-47eb-940a-e087874307df admin admin] PATH_INFO: `/v3/projects/be51ecbe872f4b1bb16db228d66ed3f7/users/1abe51b6e014434785c5440be52e8e2f/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:01.236371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697c968-675c-47eb-940a-e087874307df admin admin] RBAC: Authorizing `identity:create_grant(project_id=be51ecbe872f4b1bb16db228d66ed3f7, user_id=1abe51b6e014434785c5440be52e8e2f, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:01.239366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697c968-675c-47eb-940a-e087874307df admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:01.301930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:01.337087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1068/2116] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:01 2022] PUT /identity/v3/projects/be51ecbe872f4b1bb16db228d66ed3f7/users/1abe51b6e014434785c5440be52e8e2f/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:01.341297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed823f58-d737-45f4-8d3a-b01acce0b854 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:01.341781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed823f58-d737-45f4-8d3a-b01acce0b854 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:01.342292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed823f58-d737-45f4-8d3a-b01acce0b854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:01.364289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ed823f58-d737-45f4-8d3a-b01acce0b854 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:01.366059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ed823f58-d737-45f4-8d3a-b01acce0b854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:01.488502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1049/2117] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:01.501210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697c968-675c-47eb-940a-e087874307df admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:01.516680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5697c968-675c-47eb-940a-e087874307df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:01.642728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697c968-675c-47eb-940a-e087874307df admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:01.652212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a68af6-cf34-45e8-9ed0-2b182ba085f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:01.653227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a68af6-cf34-45e8-9ed0-2b182ba085f8 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:01.653676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a68af6-cf34-45e8-9ed0-2b182ba085f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:01.654406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a68af6-cf34-45e8-9ed0-2b182ba085f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:01.668723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a68af6-cf34-45e8-9ed0-2b182ba085f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:01.670578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a68af6-cf34-45e8-9ed0-2b182ba085f8 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:01.690628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1069/2118] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:01.924547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:02.308424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:02.593486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:03.532493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:03.588438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:03.779573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07019ef-01de-485d-884b-578146f6f142 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:03.780144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07019ef-01de-485d-884b-578146f6f142 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:03.780587 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07019ef-01de-485d-884b-578146f6f142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:03.814493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e07019ef-01de-485d-884b-578146f6f142 None None] MFA Rules not processed for user `1abe51b6e014434785c5440be52e8e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:03.816006 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e07019ef-01de-485d-884b-578146f6f142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:03.890690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1050/2119] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:03.895791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a68af6-cf34-45e8-9ed0-2b182ba085f8 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:03.908746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a68af6-cf34-45e8-9ed0-2b182ba085f8 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:03.913487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866bdda-739a-4f3c-be03-18647a373f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:03.914334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866bdda-739a-4f3c-be03-18647a373f19 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:03.914608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866bdda-739a-4f3c-be03-18647a373f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:03.914880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866bdda-739a-4f3c-be03-18647a373f19 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:03.915702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866bdda-739a-4f3c-be03-18647a373f19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:03.918160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866bdda-739a-4f3c-be03-18647a373f19 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:03.997675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1070/2120] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:03 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:04.000011 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07019ef-01de-485d-884b-578146f6f142 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:04.025012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07019ef-01de-485d-884b-578146f6f142 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:04.108229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8a7b6-33fe-4c7e-a5ec-005c384800ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:04.108845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8a7b6-33fe-4c7e-a5ec-005c384800ea admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:04.109438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8a7b6-33fe-4c7e-a5ec-005c384800ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:04.109883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8a7b6-33fe-4c7e-a5ec-005c384800ea admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:04.110522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8a7b6-33fe-4c7e-a5ec-005c384800ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:04.112736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8a7b6-33fe-4c7e-a5ec-005c384800ea admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:04.189251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1051/2121] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:03 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:04.193510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866bdda-739a-4f3c-be03-18647a373f19 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:04.211855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866bdda-739a-4f3c-be03-18647a373f19 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:04.230406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152243d9-400e-48fe-a99c-97cb6d773d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:04.230907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152243d9-400e-48fe-a99c-97cb6d773d52 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:04.231185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152243d9-400e-48fe-a99c-97cb6d773d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:04.232263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152243d9-400e-48fe-a99c-97cb6d773d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:04.232934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152243d9-400e-48fe-a99c-97cb6d773d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:04.234573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152243d9-400e-48fe-a99c-97cb6d773d52 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:04.254048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1071/2122] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:04.255120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8a7b6-33fe-4c7e-a5ec-005c384800ea admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:04.279251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:04.281390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8a7b6-33fe-4c7e-a5ec-005c384800ea admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:04.286913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6c381-a2e9-4633-b668-31b0143f92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:04.287606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6c381-a2e9-4633-b668-31b0143f92ae admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:04.287890 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6c381-a2e9-4633-b668-31b0143f92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:04.288174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6c381-a2e9-4633-b668-31b0143f92ae admin admin] PATH_INFO: `/v3/projects/05c026c9ca9640648330cfd913112bf0/users/0b8754f1df294dceaaf7aead0570c625/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:04.292722 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:04.309857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6c381-a2e9-4633-b668-31b0143f92ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c026c9ca9640648330cfd913112bf0, user_id=0b8754f1df294dceaaf7aead0570c625, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:04.313407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6c381-a2e9-4633-b668-31b0143f92ae admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:04.415396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1052/2123] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:04 2022] PUT /identity/v3/projects/05c026c9ca9640648330cfd913112bf0/users/0b8754f1df294dceaaf7aead0570c625/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:04.423297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152243d9-400e-48fe-a99c-97cb6d773d52 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:04.441362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152243d9-400e-48fe-a99c-97cb6d773d52 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:04.556302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97146965-a026-4ad0-9604-62134fd4fbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:04.557459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146965-a026-4ad0-9604-62134fd4fbdd admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:04.557745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146965-a026-4ad0-9604-62134fd4fbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:04.558012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146965-a026-4ad0-9604-62134fd4fbdd admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:04.559665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97146965-a026-4ad0-9604-62134fd4fbdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:04.561600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97146965-a026-4ad0-9604-62134fd4fbdd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:04.573646 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1072/2124] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:04.577933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6c381-a2e9-4633-b668-31b0143f92ae admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:04.590450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6c381-a2e9-4633-b668-31b0143f92ae admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:04.600658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06aab30-fd8c-46ab-9f02-80830de0d772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:04.614297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06aab30-fd8c-46ab-9f02-80830de0d772 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:04.614297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06aab30-fd8c-46ab-9f02-80830de0d772 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:04.614297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06aab30-fd8c-46ab-9f02-80830de0d772 admin admin] PATH_INFO: `/v3/projects/05c026c9ca9640648330cfd913112bf0/users/0b8754f1df294dceaaf7aead0570c625/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:04.622156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06aab30-fd8c-46ab-9f02-80830de0d772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c026c9ca9640648330cfd913112bf0, user_id=0b8754f1df294dceaaf7aead0570c625, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:04.631813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06aab30-fd8c-46ab-9f02-80830de0d772 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:04.643564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1053/2125] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:04 2022] PUT /identity/v3/projects/05c026c9ca9640648330cfd913112bf0/users/0b8754f1df294dceaaf7aead0570c625/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:05.184498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:05.283553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:05.885707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c2ab6-82fc-49d9-b0e6-301a23c19ac9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:05.898294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c2ab6-82fc-49d9-b0e6-301a23c19ac9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:05.898294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c2ab6-82fc-49d9-b0e6-301a23c19ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:05.954161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8f4c2ab6-82fc-49d9-b0e6-301a23c19ac9 None None] MFA Rules not processed for user `8da410d2ea0740dc8eeea87244fc1408`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:05.955440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8f4c2ab6-82fc-49d9-b0e6-301a23c19ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:06.067779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1073/2126] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:06.082380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06aab30-fd8c-46ab-9f02-80830de0d772 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:06.084372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e06aab30-fd8c-46ab-9f02-80830de0d772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:06.186544 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06aab30-fd8c-46ab-9f02-80830de0d772 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:06.194753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643e4b9-f684-4d20-b0f8-b74efa44e1fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:06.195431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643e4b9-f684-4d20-b0f8-b74efa44e1fc service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:06.195894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643e4b9-f684-4d20-b0f8-b74efa44e1fc service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:06.196317 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643e4b9-f684-4d20-b0f8-b74efa44e1fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:06.208000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643e4b9-f684-4d20-b0f8-b74efa44e1fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:06.210199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643e4b9-f684-4d20-b0f8-b74efa44e1fc service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:06.225226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1054/2127] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:06.268286 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2860e-d151-417a-86e9-e4a3de76db85 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:06.269222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2860e-d151-417a-86e9-e4a3de76db85 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:06.269715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2860e-d151-417a-86e9-e4a3de76db85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:06.321780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a9a2860e-d151-417a-86e9-e4a3de76db85 None None] MFA Rules not processed for user `0b8754f1df294dceaaf7aead0570c625`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:06.323761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a9a2860e-d151-417a-86e9-e4a3de76db85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:06.413188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1074/2128] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:06.425463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643e4b9-f684-4d20-b0f8-b74efa44e1fc service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:06.430097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2643e4b9-f684-4d20-b0f8-b74efa44e1fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:06.527213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643e4b9-f684-4d20-b0f8-b74efa44e1fc service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:06.531027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a496a-3f89-45cb-9678-e467b5c7029d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:06.531648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a496a-3f89-45cb-9678-e467b5c7029d service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:06.531922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a496a-3f89-45cb-9678-e467b5c7029d service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:06.532191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a496a-3f89-45cb-9678-e467b5c7029d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:06.631002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a496a-3f89-45cb-9678-e467b5c7029d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:06.633093 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a496a-3f89-45cb-9678-e467b5c7029d service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:06.643765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1055/2129] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:06.788067 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:06.821884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:08.326765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:08.691514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2860e-d151-417a-86e9-e4a3de76db85 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:08.701977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2860e-d151-417a-86e9-e4a3de76db85 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:08.706807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18130d7d-8cd8-40dc-ac6e-c45d9c633b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:08.707540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18130d7d-8cd8-40dc-ac6e-c45d9c633b5a service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:08.708084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18130d7d-8cd8-40dc-ac6e-c45d9c633b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:08.708519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18130d7d-8cd8-40dc-ac6e-c45d9c633b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:08.717641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:08.722126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18130d7d-8cd8-40dc-ac6e-c45d9c633b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:08.723817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18130d7d-8cd8-40dc-ac6e-c45d9c633b5a service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:08.748159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1075/2130] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:08.928124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ecfc9-6320-40c1-8303-b2a2441b290c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:08.928564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ecfc9-6320-40c1-8303-b2a2441b290c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:08.928902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ecfc9-6320-40c1-8303-b2a2441b290c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:08.954151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-7a7ecfc9-6320-40c1-8303-b2a2441b290c None None] MFA Rules not processed for user `8da410d2ea0740dc8eeea87244fc1408`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:08.969319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7a7ecfc9-6320-40c1-8303-b2a2441b290c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:08.990862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1056/2131] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:09.753374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:19.707470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=18 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:20.350892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18130d7d-8cd8-40dc-ac6e-c45d9c633b5a service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:20.352787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-18130d7d-8cd8-40dc-ac6e-c45d9c633b5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:20.438916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18130d7d-8cd8-40dc-ac6e-c45d9c633b5a service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:20.451934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:20.454566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:20.454566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:20.454566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] PATH_INFO: `/v3/users/60cdd6ffe8324569b78cd91d82df9e42` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:20.455538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60cdd6ffe8324569b78cd91d82df9e42)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:20.457746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:20.517241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=81 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:20.643758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60cdd6ffe8324569b78cd91d82df9e42'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:20.644528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60cdd6ffe8324569b78cd91d82df9e42'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:20.645198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60cdd6ffe8324569b78cd91d82df9e42'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:20.658159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60cdd6ffe8324569b78cd91d82df9e42'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:20.683524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60cdd6ffe8324569b78cd91d82df9e42'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:20.746783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1076/2132] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:20 2022] DELETE /identity/v3/users/60cdd6ffe8324569b78cd91d82df9e42 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:20.751360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16160f16-d059-4c92-8997-003701aad82a None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:20.751650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16160f16-d059-4c92-8997-003701aad82a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:20.751930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16160f16-d059-4c92-8997-003701aad82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:20.772453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-16160f16-d059-4c92-8997-003701aad82a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:20.773743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-16160f16-d059-4c92-8997-003701aad82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:20.850773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1057/2133] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:20.857564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:20.859952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:20.924111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eaa57-7323-457b-9504-0923b48ed131 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:20.927786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33356d5-d84f-451e-a83f-e52c764911be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:20.928385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33356d5-d84f-451e-a83f-e52c764911be service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:20.928664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33356d5-d84f-451e-a83f-e52c764911be service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:20.928939 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33356d5-d84f-451e-a83f-e52c764911be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:20.938991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33356d5-d84f-451e-a83f-e52c764911be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:20.940461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33356d5-d84f-451e-a83f-e52c764911be service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:20.951287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1077/2134] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:21.502368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16160f16-d059-4c92-8997-003701aad82a None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:21.504066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-16160f16-d059-4c92-8997-003701aad82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:21.525552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16160f16-d059-4c92-8997-003701aad82a None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:21.533232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:21.534152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:21.534487 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:21.534862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] PATH_INFO: `/v3/projects/1e6b50c9b12e495f89c0ba254957522d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:21.536444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6b50c9b12e495f89c0ba254957522d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:21.539449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:21.702441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6b50c9b12e495f89c0ba254957522d', 'request_id': 'req-65d6e9ac-39cd-4ab3-8523-52308b05b252'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:21.712167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6b50c9b12e495f89c0ba254957522d', 'request_id': 'req-65d6e9ac-39cd-4ab3-8523-52308b05b252'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:21.719498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6b50c9b12e495f89c0ba254957522d', 'request_id': 'req-65d6e9ac-39cd-4ab3-8523-52308b05b252'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:21.821803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] The token cache is being invalidate because project 1e6b50c9b12e495f89c0ba254957522d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:21.822386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:21.823367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-65d6e9ac-39cd-4ab3-8523-52308b05b252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:21.825102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1058/2135] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:21 2022] DELETE /identity/v3/projects/1e6b50c9b12e495f89c0ba254957522d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:21.830834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76328c8-0412-4f84-9ab3-8b5fab690e1a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:21.831399 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76328c8-0412-4f84-9ab3-8b5fab690e1a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:21.831824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76328c8-0412-4f84-9ab3-8b5fab690e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:21.855114 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a76328c8-0412-4f84-9ab3-8b5fab690e1a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:21.856343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a76328c8-0412-4f84-9ab3-8b5fab690e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:21.878665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:21.954538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1078/2136] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:21.963250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef0d16-1eab-45dc-ab1b-830ea96f3784 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:21.963590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef0d16-1eab-45dc-ab1b-830ea96f3784 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:21.963872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef0d16-1eab-45dc-ab1b-830ea96f3784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:21.988347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b4ef0d16-1eab-45dc-ab1b-830ea96f3784 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:21.989847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b4ef0d16-1eab-45dc-ab1b-830ea96f3784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:22.003907 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1059/2137] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:22.008629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76328c8-0412-4f84-9ab3-8b5fab690e1a None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:22.020201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76328c8-0412-4f84-9ab3-8b5fab690e1a None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:22.025329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44ca62-5fd8-420b-a26e-d3eb13d4b739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:22.026245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44ca62-5fd8-420b-a26e-d3eb13d4b739 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:22.026612 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44ca62-5fd8-420b-a26e-d3eb13d4b739 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:22.027158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44ca62-5fd8-420b-a26e-d3eb13d4b739 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:22.027701 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44ca62-5fd8-420b-a26e-d3eb13d4b739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:53:22.028087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44ca62-5fd8-420b-a26e-d3eb13d4b739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:22.029951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44ca62-5fd8-420b-a26e-d3eb13d4b739 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:22.055507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1079/2138] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:53:22.055997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef0d16-1eab-45dc-ab1b-830ea96f3784 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:22.078687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef0d16-1eab-45dc-ab1b-830ea96f3784 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:22.091229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381a0c6-bb3d-4ee1-8c13-6cba8cfb877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:22.091867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381a0c6-bb3d-4ee1-8c13-6cba8cfb877a admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:22.092151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381a0c6-bb3d-4ee1-8c13-6cba8cfb877a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:22.092428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381a0c6-bb3d-4ee1-8c13-6cba8cfb877a admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:22.093377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381a0c6-bb3d-4ee1-8c13-6cba8cfb877a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:22.095371 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381a0c6-bb3d-4ee1-8c13-6cba8cfb877a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:22.185092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1060/2139] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:22 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:22.189568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44ca62-5fd8-420b-a26e-d3eb13d4b739 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:22.202241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44ca62-5fd8-420b-a26e-d3eb13d4b739 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:22.278364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57a473-a3a8-444c-8ae4-0d4aca3fd4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:22.279132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57a473-a3a8-444c-8ae4-0d4aca3fd4be admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:22.279487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57a473-a3a8-444c-8ae4-0d4aca3fd4be admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:22.279837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57a473-a3a8-444c-8ae4-0d4aca3fd4be admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:22.280552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57a473-a3a8-444c-8ae4-0d4aca3fd4be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:22.282872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57a473-a3a8-444c-8ae4-0d4aca3fd4be admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:22.387486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1080/2140] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:22 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:22.391543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381a0c6-bb3d-4ee1-8c13-6cba8cfb877a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:22.407407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381a0c6-bb3d-4ee1-8c13-6cba8cfb877a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:22.412707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500834cc-b72a-49df-94bb-c34029b72a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:22.413378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500834cc-b72a-49df-94bb-c34029b72a32 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:22.413793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500834cc-b72a-49df-94bb-c34029b72a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:22.413937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500834cc-b72a-49df-94bb-c34029b72a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:22.414630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500834cc-b72a-49df-94bb-c34029b72a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:22.416283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500834cc-b72a-49df-94bb-c34029b72a32 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:22.431757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1061/2141] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:53:22.435920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57a473-a3a8-444c-8ae4-0d4aca3fd4be admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:22.443421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57a473-a3a8-444c-8ae4-0d4aca3fd4be admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:22.448578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c414eb-b9bd-4d67-ab68-338b69d83684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:22.449241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c414eb-b9bd-4d67-ab68-338b69d83684 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:22.449550 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c414eb-b9bd-4d67-ab68-338b69d83684 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:22.449826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c414eb-b9bd-4d67-ab68-338b69d83684 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:22.450470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c414eb-b9bd-4d67-ab68-338b69d83684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:22.452049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c414eb-b9bd-4d67-ab68-338b69d83684 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:22.465611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1081/2142] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:22.469764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500834cc-b72a-49df-94bb-c34029b72a32 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:22.480952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500834cc-b72a-49df-94bb-c34029b72a32 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:22.485723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457ee35-19e7-4afb-9113-9a6a0481d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:22.486664 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457ee35-19e7-4afb-9113-9a6a0481d6ac admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:22.486951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457ee35-19e7-4afb-9113-9a6a0481d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:22.487234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457ee35-19e7-4afb-9113-9a6a0481d6ac admin admin] PATH_INFO: `/v3/projects/feb31c5abc0c4ddfb54181b5ee2a2f59/users/3714c5827b2d4eeaadb7830e8b662813/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:22.506483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457ee35-19e7-4afb-9113-9a6a0481d6ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb31c5abc0c4ddfb54181b5ee2a2f59, user_id=3714c5827b2d4eeaadb7830e8b662813, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:22.509224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457ee35-19e7-4afb-9113-9a6a0481d6ac admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:22.612664 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1062/2143] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:22 2022] PUT /identity/v3/projects/feb31c5abc0c4ddfb54181b5ee2a2f59/users/3714c5827b2d4eeaadb7830e8b662813/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:22.623498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff566a51-918c-47dd-88d2-70293f887cd7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:22.623781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff566a51-918c-47dd-88d2-70293f887cd7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:22.624053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff566a51-918c-47dd-88d2-70293f887cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:22.654480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-ff566a51-918c-47dd-88d2-70293f887cd7 None None] MFA Rules not processed for user `3714c5827b2d4eeaadb7830e8b662813`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:22.655726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ff566a51-918c-47dd-88d2-70293f887cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:22.719921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1082/2144] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:22.724557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457ee35-19e7-4afb-9113-9a6a0481d6ac admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:22.754236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:22.801265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457ee35-19e7-4afb-9113-9a6a0481d6ac admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:22.806179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a2f25-0f50-43b1-b034-236ff4fdceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:22.806787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a2f25-0f50-43b1-b034-236ff4fdceb1 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:22.807117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a2f25-0f50-43b1-b034-236ff4fdceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:22.807449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a2f25-0f50-43b1-b034-236ff4fdceb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:22.808185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a2f25-0f50-43b1-b034-236ff4fdceb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:22.810176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a2f25-0f50-43b1-b034-236ff4fdceb1 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:22.897341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1063/2145] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:22 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:22.902021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff566a51-918c-47dd-88d2-70293f887cd7 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:22.919207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff566a51-918c-47dd-88d2-70293f887cd7 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:22.931712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:22.977935 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d1e4c-40af-4630-93c6-fa80b7777926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:22.978556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1e4c-40af-4630-93c6-fa80b7777926 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:22.978946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1e4c-40af-4630-93c6-fa80b7777926 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:22.979922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d1e4c-40af-4630-93c6-fa80b7777926 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:22.980624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d1e4c-40af-4630-93c6-fa80b7777926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:22.982572 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d1e4c-40af-4630-93c6-fa80b7777926 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:23.098368 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1083/2146] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:22 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:23.099114 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a2f25-0f50-43b1-b034-236ff4fdceb1 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:23.109849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a2f25-0f50-43b1-b034-236ff4fdceb1 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:23.115747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba3087-0c49-40b1-8959-7ab4e8c8cf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:23.115747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba3087-0c49-40b1-8959-7ab4e8c8cf9d admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:23.116335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba3087-0c49-40b1-8959-7ab4e8c8cf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:23.116706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba3087-0c49-40b1-8959-7ab4e8c8cf9d admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:23.117506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba3087-0c49-40b1-8959-7ab4e8c8cf9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:23.119254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba3087-0c49-40b1-8959-7ab4e8c8cf9d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:23.132015 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1064/2147] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:23.137815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d1e4c-40af-4630-93c6-fa80b7777926 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:23.152884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d1e4c-40af-4630-93c6-fa80b7777926 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:23.154561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8402e2-8ab7-4526-a02a-df6f215d849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:23.155196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8402e2-8ab7-4526-a02a-df6f215d849e admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:23.155506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8402e2-8ab7-4526-a02a-df6f215d849e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:23.155782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8402e2-8ab7-4526-a02a-df6f215d849e admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:23.156433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8402e2-8ab7-4526-a02a-df6f215d849e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:23.158122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8402e2-8ab7-4526-a02a-df6f215d849e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:23.173304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1084/2148] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:23.174485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba3087-0c49-40b1-8959-7ab4e8c8cf9d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:23.186351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba3087-0c49-40b1-8959-7ab4e8c8cf9d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:23.191946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd08f95-dfe9-4fc8-a3fb-c3f7eb056ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:23.192614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd08f95-dfe9-4fc8-a3fb-c3f7eb056ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:23.192893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd08f95-dfe9-4fc8-a3fb-c3f7eb056ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:23.193291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd08f95-dfe9-4fc8-a3fb-c3f7eb056ecb admin admin] PATH_INFO: `/v3/projects/33b4cf1e8e30452280dacb3082f957f0/users/e29b6f8c4bf54d1d8fe0b2bd1828d7b6/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:23.211067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd08f95-dfe9-4fc8-a3fb-c3f7eb056ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b4cf1e8e30452280dacb3082f957f0, user_id=e29b6f8c4bf54d1d8fe0b2bd1828d7b6, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:23.228517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd08f95-dfe9-4fc8-a3fb-c3f7eb056ecb admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:23.335894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1065/2149] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:23 2022] PUT /identity/v3/projects/33b4cf1e8e30452280dacb3082f957f0/users/e29b6f8c4bf54d1d8fe0b2bd1828d7b6/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:23.348982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5330755-0430-434b-ac86-aac8916a5d55 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:23.349311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5330755-0430-434b-ac86-aac8916a5d55 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:23.349591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5330755-0430-434b-ac86-aac8916a5d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:23.379650 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f5330755-0430-434b-ac86-aac8916a5d55 None None] MFA Rules not processed for user `e29b6f8c4bf54d1d8fe0b2bd1828d7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:23.380981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f5330755-0430-434b-ac86-aac8916a5d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:23.456683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1085/2150] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:23.461365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd08f95-dfe9-4fc8-a3fb-c3f7eb056ecb admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:23.535000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd08f95-dfe9-4fc8-a3fb-c3f7eb056ecb admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:23.539951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e835ad-56d4-4103-91b3-a3cf28a50220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:23.540606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e835ad-56d4-4103-91b3-a3cf28a50220 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:23.540891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e835ad-56d4-4103-91b3-a3cf28a50220 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:23.541269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e835ad-56d4-4103-91b3-a3cf28a50220 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:23.542725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e835ad-56d4-4103-91b3-a3cf28a50220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:23.544642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e835ad-56d4-4103-91b3-a3cf28a50220 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:23.640268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1066/2151] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:23 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:23.644289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5330755-0430-434b-ac86-aac8916a5d55 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:23.656116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5330755-0430-434b-ac86-aac8916a5d55 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:23.711915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50ad8a-148f-4c65-9778-a15ca834b9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:23.712438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50ad8a-148f-4c65-9778-a15ca834b9cd admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:23.712714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50ad8a-148f-4c65-9778-a15ca834b9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:23.712984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50ad8a-148f-4c65-9778-a15ca834b9cd admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:23.714288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50ad8a-148f-4c65-9778-a15ca834b9cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:23.716188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50ad8a-148f-4c65-9778-a15ca834b9cd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:23.821484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1086/2152] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:23 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:23.826270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e835ad-56d4-4103-91b3-a3cf28a50220 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:23.837328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e835ad-56d4-4103-91b3-a3cf28a50220 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:23.842740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246fab1-ff5d-496a-beeb-7cdf23e7d86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:23.843340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246fab1-ff5d-496a-beeb-7cdf23e7d86e admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:23.843631 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246fab1-ff5d-496a-beeb-7cdf23e7d86e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:23.843920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246fab1-ff5d-496a-beeb-7cdf23e7d86e admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:23.844560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246fab1-ff5d-496a-beeb-7cdf23e7d86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:23.846194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246fab1-ff5d-496a-beeb-7cdf23e7d86e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:23.856967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1067/2153] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:23.867556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50ad8a-148f-4c65-9778-a15ca834b9cd admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:23.877645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50ad8a-148f-4c65-9778-a15ca834b9cd admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:23.882509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3b5d3-e1d5-48e8-b310-5a84eb469dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:23.883165 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b5d3-e1d5-48e8-b310-5a84eb469dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:23.883447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b5d3-e1d5-48e8-b310-5a84eb469dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:23.883725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b5d3-e1d5-48e8-b310-5a84eb469dbd admin admin] PATH_INFO: `/v3/projects/05533c7555634b208300e8f1a380ebfb/users/2defe44cb47e41eebb9294a28aa7e4bc/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:23.898902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:23.909594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3b5d3-e1d5-48e8-b310-5a84eb469dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=05533c7555634b208300e8f1a380ebfb, user_id=2defe44cb47e41eebb9294a28aa7e4bc, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:23.909594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3b5d3-e1d5-48e8-b310-5a84eb469dbd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:24.055484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1087/2154] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:23 2022] PUT /identity/v3/projects/05533c7555634b208300e8f1a380ebfb/users/2defe44cb47e41eebb9294a28aa7e4bc/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:24.060626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246fab1-ff5d-496a-beeb-7cdf23e7d86e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:24.076855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246fab1-ff5d-496a-beeb-7cdf23e7d86e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:24.121213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:24.136885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd307725-4422-4570-aaf8-e04553302c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:24.137557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd307725-4422-4570-aaf8-e04553302c41 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:24.137839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd307725-4422-4570-aaf8-e04553302c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:24.138114 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd307725-4422-4570-aaf8-e04553302c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:24.138805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd307725-4422-4570-aaf8-e04553302c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:24.140432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd307725-4422-4570-aaf8-e04553302c41 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:24.152235 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1068/2155] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:24.157267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3b5d3-e1d5-48e8-b310-5a84eb469dbd admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:24.168062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3b5d3-e1d5-48e8-b310-5a84eb469dbd admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:24.173692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665d6f5-37ab-454b-88c4-298b8378ab66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:24.174363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665d6f5-37ab-454b-88c4-298b8378ab66 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:24.174638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665d6f5-37ab-454b-88c4-298b8378ab66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:24.174920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665d6f5-37ab-454b-88c4-298b8378ab66 admin admin] PATH_INFO: `/v3/projects/05533c7555634b208300e8f1a380ebfb/users/2defe44cb47e41eebb9294a28aa7e4bc/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:24.176890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665d6f5-37ab-454b-88c4-298b8378ab66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05533c7555634b208300e8f1a380ebfb, user_id=2defe44cb47e41eebb9294a28aa7e4bc, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:24.179577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665d6f5-37ab-454b-88c4-298b8378ab66 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:24.191304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1088/2156] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:24 2022] PUT /identity/v3/projects/05533c7555634b208300e8f1a380ebfb/users/2defe44cb47e41eebb9294a28aa7e4bc/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:24.201996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cfbe7-b24d-417c-a489-98a5e0b2dbe9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:24.202287 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cfbe7-b24d-417c-a489-98a5e0b2dbe9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:24.202638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cfbe7-b24d-417c-a489-98a5e0b2dbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:24.240983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b53cfbe7-b24d-417c-a489-98a5e0b2dbe9 None None] MFA Rules not processed for user `2defe44cb47e41eebb9294a28aa7e4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:24.242290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b53cfbe7-b24d-417c-a489-98a5e0b2dbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:24.353002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1069/2157] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:24.370798 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665d6f5-37ab-454b-88c4-298b8378ab66 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:24.372760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c665d6f5-37ab-454b-88c4-298b8378ab66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:24.449810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665d6f5-37ab-454b-88c4-298b8378ab66 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:24.455658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b8b49-3235-455f-b345-8f0232843497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:24.456346 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b8b49-3235-455f-b345-8f0232843497 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:24.456625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b8b49-3235-455f-b345-8f0232843497 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:24.456895 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b8b49-3235-455f-b345-8f0232843497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:24.517652 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b8b49-3235-455f-b345-8f0232843497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:24.519089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b8b49-3235-455f-b345-8f0232843497 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:24.533562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1089/2158] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:26.561907 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53cfbe7-b24d-417c-a489-98a5e0b2dbe9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:26.574245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53cfbe7-b24d-417c-a489-98a5e0b2dbe9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:26.584501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d942457-7e04-441b-a59e-708dcc7bbe02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:26.584501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d942457-7e04-441b-a59e-708dcc7bbe02 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:26.584501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d942457-7e04-441b-a59e-708dcc7bbe02 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:26.584501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d942457-7e04-441b-a59e-708dcc7bbe02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:26.667317 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d942457-7e04-441b-a59e-708dcc7bbe02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:26.668779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d942457-7e04-441b-a59e-708dcc7bbe02 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:26.678342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1070/2159] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:28.321393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b8b49-3235-455f-b345-8f0232843497 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:28.323890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-934b8b49-3235-455f-b345-8f0232843497 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:28.339698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b8b49-3235-455f-b345-8f0232843497 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:28.343617 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62a564-3865-40e9-ba68-e3d19f7f9047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:28.344423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62a564-3865-40e9-ba68-e3d19f7f9047 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:28.344711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62a564-3865-40e9-ba68-e3d19f7f9047 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:28.345016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62a564-3865-40e9-ba68-e3d19f7f9047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:28.355742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62a564-3865-40e9-ba68-e3d19f7f9047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:28.357431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62a564-3865-40e9-ba68-e3d19f7f9047 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:28.368592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1090/2160] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:28.721946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d942457-7e04-441b-a59e-708dcc7bbe02 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:28.723385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0d942457-7e04-441b-a59e-708dcc7bbe02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:28.793520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d942457-7e04-441b-a59e-708dcc7bbe02 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:28.798649 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:28.799309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:28.799591 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:28.799983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] PATH_INFO: `/v3/users/1abe51b6e014434785c5440be52e8e2f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:28.801626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abe51b6e014434785c5440be52e8e2f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:28.803940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:28.810171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:29.013665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abe51b6e014434785c5440be52e8e2f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.014334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abe51b6e014434785c5440be52e8e2f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.014950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abe51b6e014434785c5440be52e8e2f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.027322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abe51b6e014434785c5440be52e8e2f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.034785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abe51b6e014434785c5440be52e8e2f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.119218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1071/2161] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:28 2022] DELETE /identity/v3/users/1abe51b6e014434785c5440be52e8e2f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:29.325683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62a564-3865-40e9-ba68-e3d19f7f9047 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:29.327098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-fe62a564-3865-40e9-ba68-e3d19f7f9047 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:29.340900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62a564-3865-40e9-ba68-e3d19f7f9047 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:29.406138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:29.406820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:29.407517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:29.407517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] PATH_INFO: `/v3/users/16c9ec1a00414f9db636b13c169af63b` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:29.408704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c9ec1a00414f9db636b13c169af63b)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:29.410808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:29.435389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:29.485904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:29.487605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:29.504197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474253b8-cfe5-4019-b3de-06872a660e24 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:29.509128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:29.509781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:29.510137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:29.510420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] PATH_INFO: `/v3/projects/be51ecbe872f4b1bb16db228d66ed3f7` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:29.511621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be51ecbe872f4b1bb16db228d66ed3f7)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:29.513608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:29.627454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c9ec1a00414f9db636b13c169af63b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.628137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c9ec1a00414f9db636b13c169af63b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.629148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c9ec1a00414f9db636b13c169af63b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.638582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c9ec1a00414f9db636b13c169af63b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.645630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be51ecbe872f4b1bb16db228d66ed3f7', 'request_id': 'req-89ad7daa-f910-488b-9f95-d56a0182e1c7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.655638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be51ecbe872f4b1bb16db228d66ed3f7', 'request_id': 'req-89ad7daa-f910-488b-9f95-d56a0182e1c7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.669934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-2d5858d4-622a-4945-bea0-b34eb6b53293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c9ec1a00414f9db636b13c169af63b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.678126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be51ecbe872f4b1bb16db228d66ed3f7', 'request_id': 'req-89ad7daa-f910-488b-9f95-d56a0182e1c7'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.793890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1091/2162] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:29 2022] DELETE /identity/v3/users/16c9ec1a00414f9db636b13c169af63b => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:29.798306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf73ee-ed28-49c8-8cbc-4770e46984dc None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:29.798597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf73ee-ed28-49c8-8cbc-4770e46984dc None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:29.798874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf73ee-ed28-49c8-8cbc-4770e46984dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:29.824075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-becf73ee-ed28-49c8-8cbc-4770e46984dc None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:29.826133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-becf73ee-ed28-49c8-8cbc-4770e46984dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:29.860449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] The token cache is being invalidate because project be51ecbe872f4b1bb16db228d66ed3f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:29.861387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.862408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:29.864033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1072/2163] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:29 2022] DELETE /identity/v3/projects/be51ecbe872f4b1bb16db228d66ed3f7 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:29.867991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:29.871992 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:29.914137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1092/2164] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:29.930931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf73ee-ed28-49c8-8cbc-4770e46984dc None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:29.940029 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-becf73ee-ed28-49c8-8cbc-4770e46984dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:29.994502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad7daa-f910-488b-9f95-d56a0182e1c7 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:30.000459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:30.001144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:30.001433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:30.001877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] PATH_INFO: `/v3/users/0b8754f1df294dceaaf7aead0570c625` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:30.003271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8754f1df294dceaaf7aead0570c625)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:30.005411 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:30.014216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:30.036496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf73ee-ed28-49c8-8cbc-4770e46984dc None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:30.042359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efcab8-b998-4d9f-89cd-8f28be005dfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:30.043611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efcab8-b998-4d9f-89cd-8f28be005dfd service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:30.044082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efcab8-b998-4d9f-89cd-8f28be005dfd service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:30.044507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efcab8-b998-4d9f-89cd-8f28be005dfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:30.054664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efcab8-b998-4d9f-89cd-8f28be005dfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:30.065249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22efcab8-b998-4d9f-89cd-8f28be005dfd service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:30.066758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1093/2165] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:30.230041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8754f1df294dceaaf7aead0570c625'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:30.230989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8754f1df294dceaaf7aead0570c625'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:30.231688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8754f1df294dceaaf7aead0570c625'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:30.246216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8754f1df294dceaaf7aead0570c625'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:30.254493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9c493621-ec32-4152-b3b7-83a14d15d326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8754f1df294dceaaf7aead0570c625'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:30.355588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1073/2166] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:29 2022] DELETE /identity/v3/users/0b8754f1df294dceaaf7aead0570c625 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:30.674846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efcab8-b998-4d9f-89cd-8f28be005dfd service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:30.687526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efcab8-b998-4d9f-89cd-8f28be005dfd service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:30.749885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:30.766981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:30.767660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:30.767964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:30.768239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] PATH_INFO: `/v3/projects/05c026c9ca9640648330cfd913112bf0` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:30.769422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c026c9ca9640648330cfd913112bf0)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:30.771359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:30.892503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c026c9ca9640648330cfd913112bf0', 'request_id': 'req-158cf482-0a7f-46e4-bac4-e5d3bd74f375'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:30.908443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c026c9ca9640648330cfd913112bf0', 'request_id': 'req-158cf482-0a7f-46e4-bac4-e5d3bd74f375'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:30.977025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c026c9ca9640648330cfd913112bf0', 'request_id': 'req-158cf482-0a7f-46e4-bac4-e5d3bd74f375'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:31.022982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] The token cache is being invalidate because project 05c026c9ca9640648330cfd913112bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:31.022982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:31.022982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:31.024667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1094/2167] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:30 2022] DELETE /identity/v3/projects/05c026c9ca9640648330cfd913112bf0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:31.030495 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4b18e-2f75-4e0c-9a92-7a57eb31d5a9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.031389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4b18e-2f75-4e0c-9a92-7a57eb31d5a9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.032424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4b18e-2f75-4e0c-9a92-7a57eb31d5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.056188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e7c4b18e-2f75-4e0c-9a92-7a57eb31d5a9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:31.058755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e7c4b18e-2f75-4e0c-9a92-7a57eb31d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:31.091586 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:31.094115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:31.141900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1074/2168] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:31.149727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ac515-e205-4a03-a5aa-a749d0b8fca0 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.150140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ac515-e205-4a03-a5aa-a749d0b8fca0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.151922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ac515-e205-4a03-a5aa-a749d0b8fca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.183824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-6a8ac515-e205-4a03-a5aa-a749d0b8fca0 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:31.185121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cf482-0a7f-46e4-bac4-e5d3bd74f375 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:31.185331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6a8ac515-e205-4a03-a5aa-a749d0b8fca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:31.190726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:31.191443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.191838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.192153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] PATH_INFO: `/v3/projects/0c9ea7220b974db79488cbe3ec7a4b39` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.194270 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9ea7220b974db79488cbe3ec7a4b39)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:31.196306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:31.199801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1075/2169] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:31.203696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ac515-e205-4a03-a5aa-a749d0b8fca0 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:31.216693 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ac515-e205-4a03-a5aa-a749d0b8fca0 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:31.223873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd76261-257f-47f4-b56b-f5a335f40e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:31.225081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd76261-257f-47f4-b56b-f5a335f40e0f admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.225081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd76261-257f-47f4-b56b-f5a335f40e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.225266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd76261-257f-47f4-b56b-f5a335f40e0f admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.225974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd76261-257f-47f4-b56b-f5a335f40e0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:53:31.226343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd76261-257f-47f4-b56b-f5a335f40e0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:31.228502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd76261-257f-47f4-b56b-f5a335f40e0f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:31.241862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1076/2170] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:53:31.247744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd76261-257f-47f4-b56b-f5a335f40e0f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:31.258776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd76261-257f-47f4-b56b-f5a335f40e0f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:31.264884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b692f-7fbb-4020-9e4f-e5c4464108d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:31.265826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b692f-7fbb-4020-9e4f-e5c4464108d4 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.265826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b692f-7fbb-4020-9e4f-e5c4464108d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.266032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b692f-7fbb-4020-9e4f-e5c4464108d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.266698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b692f-7fbb-4020-9e4f-e5c4464108d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:31.268605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b692f-7fbb-4020-9e4f-e5c4464108d4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:31.332439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9ea7220b974db79488cbe3ec7a4b39', 'request_id': 'req-9aa28a49-b487-4527-bfd4-6bff55291e5f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:31.340330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9ea7220b974db79488cbe3ec7a4b39', 'request_id': 'req-9aa28a49-b487-4527-bfd4-6bff55291e5f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:31.376070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1077/2171] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:31 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:31.377521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b692f-7fbb-4020-9e4f-e5c4464108d4 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:31.431654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9ea7220b974db79488cbe3ec7a4b39', 'request_id': 'req-9aa28a49-b487-4527-bfd4-6bff55291e5f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:31.435107 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:31.444382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] The token cache is being invalidate because project 0c9ea7220b974db79488cbe3ec7a4b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:31.444823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:31.445543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9aa28a49-b487-4527-bfd4-6bff55291e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:31.447450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1095/2172] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:31 2022] DELETE /identity/v3/projects/0c9ea7220b974db79488cbe3ec7a4b39 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:31.453860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab86ebe-aceb-4bd4-ad70-0dedf5b355df None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.454193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab86ebe-aceb-4bd4-ad70-0dedf5b355df None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.454431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab86ebe-aceb-4bd4-ad70-0dedf5b355df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.488152 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:31.511001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b692f-7fbb-4020-9e4f-e5c4464108d4 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:31.518450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4ab86ebe-aceb-4bd4-ad70-0dedf5b355df None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:31.519877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4ab86ebe-aceb-4bd4-ad70-0dedf5b355df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:31.534952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1096/2173] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:31.537385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f32f5f-b831-4543-845f-9ee86b178674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:31.539313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f32f5f-b831-4543-845f-9ee86b178674 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.539945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f32f5f-b831-4543-845f-9ee86b178674 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.540535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f32f5f-b831-4543-845f-9ee86b178674 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.542413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f32f5f-b831-4543-845f-9ee86b178674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:31.544794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f32f5f-b831-4543-845f-9ee86b178674 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:31.545809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cee9ab-3dc4-4c29-8939-e3258328fe18 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.545809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cee9ab-3dc4-4c29-8939-e3258328fe18 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.557997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cee9ab-3dc4-4c29-8939-e3258328fe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.589180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e2cee9ab-3dc4-4c29-8939-e3258328fe18 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:31.591631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e2cee9ab-3dc4-4c29-8939-e3258328fe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:31.604270 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1097/2174] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:31.612456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cee9ab-3dc4-4c29-8939-e3258328fe18 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:31.626204 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cee9ab-3dc4-4c29-8939-e3258328fe18 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:31.633724 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50f987-06dd-4dfa-acb1-88af50c0927e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:31.634583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50f987-06dd-4dfa-acb1-88af50c0927e admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.635627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50f987-06dd-4dfa-acb1-88af50c0927e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.635922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50f987-06dd-4dfa-acb1-88af50c0927e admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.636669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50f987-06dd-4dfa-acb1-88af50c0927e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:53:31.636992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50f987-06dd-4dfa-acb1-88af50c0927e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:31.638627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50f987-06dd-4dfa-acb1-88af50c0927e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:31.652521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1098/2175] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:53:31.657324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50f987-06dd-4dfa-acb1-88af50c0927e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:31.669456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50f987-06dd-4dfa-acb1-88af50c0927e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:31.678340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f7ba8-0544-4f90-bce8-4d9e2dffbbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:31.682094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7ba8-0544-4f90-bce8-4d9e2dffbbe2 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.682094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7ba8-0544-4f90-bce8-4d9e2dffbbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.682094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f7ba8-0544-4f90-bce8-4d9e2dffbbe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.682094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f7ba8-0544-4f90-bce8-4d9e2dffbbe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:31.682650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1078/2176] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:31 2022] POST /identity/v3/users => generated 292 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:31.685637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f7ba8-0544-4f90-bce8-4d9e2dffbbe2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:31.688533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f32f5f-b831-4543-845f-9ee86b178674 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:31.694911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-66f32f5f-b831-4543-845f-9ee86b178674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:31.757079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f32f5f-b831-4543-845f-9ee86b178674 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:31.767684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba291e-5dd8-4c2c-a75d-9762fad45da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:31.770746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba291e-5dd8-4c2c-a75d-9762fad45da3 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.772021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba291e-5dd8-4c2c-a75d-9762fad45da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.772653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba291e-5dd8-4c2c-a75d-9762fad45da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.773639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba291e-5dd8-4c2c-a75d-9762fad45da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:31.775436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba291e-5dd8-4c2c-a75d-9762fad45da3 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:31.795102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1099/2177] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:31 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:31.802039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f7ba8-0544-4f90-bce8-4d9e2dffbbe2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:31.804751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1079/2178] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:31.817241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba291e-5dd8-4c2c-a75d-9762fad45da3 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:31.861076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba291e-5dd8-4c2c-a75d-9762fad45da3 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:31.938118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f7ba8-0544-4f90-bce8-4d9e2dffbbe2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:31.956982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990be0c-10d8-4a0f-b992-05764875f317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:31.957664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990be0c-10d8-4a0f-b992-05764875f317 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.957950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990be0c-10d8-4a0f-b992-05764875f317 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.958226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990be0c-10d8-4a0f-b992-05764875f317 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.958890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990be0c-10d8-4a0f-b992-05764875f317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:31.972196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990be0c-10d8-4a0f-b992-05764875f317 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:31.990354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da04a9-cb89-48c4-8310-a1609781cf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:31.990993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da04a9-cb89-48c4-8310-a1609781cf33 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:31.991267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da04a9-cb89-48c4-8310-a1609781cf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:31.997188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da04a9-cb89-48c4-8310-a1609781cf33 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:31.997188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da04a9-cb89-48c4-8310-a1609781cf33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:31.997877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1100/2179] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:31.997877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990be0c-10d8-4a0f-b992-05764875f317 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:31.998247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da04a9-cb89-48c4-8310-a1609781cf33 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:32.015042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990be0c-10d8-4a0f-b992-05764875f317 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:32.026414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9240b-8a00-4ed2-95f9-86b66b177284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:32.027406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9240b-8a00-4ed2-95f9-86b66b177284 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:32.028097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9240b-8a00-4ed2-95f9-86b66b177284 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:32.028619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9240b-8a00-4ed2-95f9-86b66b177284 admin admin] PATH_INFO: `/v3/projects/252f15925b724a579851dc4df7e9efe1/users/80693946241344279bad3daed358a50b/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:32.056328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9240b-8a00-4ed2-95f9-86b66b177284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=252f15925b724a579851dc4df7e9efe1, user_id=80693946241344279bad3daed358a50b, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:32.059947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9240b-8a00-4ed2-95f9-86b66b177284 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:32.110212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1080/2180] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:31 2022] POST /identity/v3/users => generated 288 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:32.113732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da04a9-cb89-48c4-8310-a1609781cf33 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:32.132205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da04a9-cb89-48c4-8310-a1609781cf33 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:32.139775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32149b0-8705-4534-937a-030c0cadb26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:32.153187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32149b0-8705-4534-937a-030c0cadb26c admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:32.153187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32149b0-8705-4534-937a-030c0cadb26c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:32.153187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32149b0-8705-4534-937a-030c0cadb26c admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:32.153187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32149b0-8705-4534-937a-030c0cadb26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:32.153187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32149b0-8705-4534-937a-030c0cadb26c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:32.160310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1081/2181] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:32.164391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32149b0-8705-4534-937a-030c0cadb26c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:32.171510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1101/2182] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:31 2022] PUT /identity/v3/projects/252f15925b724a579851dc4df7e9efe1/users/80693946241344279bad3daed358a50b/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:32.180810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32149b0-8705-4534-937a-030c0cadb26c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:32.183559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029955e2-778a-494b-b865-e939b665008c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:32.184062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029955e2-778a-494b-b865-e939b665008c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:32.184333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029955e2-778a-494b-b865-e939b665008c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:32.225793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-029955e2-778a-494b-b865-e939b665008c None None] MFA Rules not processed for user `80693946241344279bad3daed358a50b`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:32.228597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-029955e2-778a-494b-b865-e939b665008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:32.260863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7ebca-8ff3-480b-8c99-b867ce4a55e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:32.261945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7ebca-8ff3-480b-8c99-b867ce4a55e0 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:32.262232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7ebca-8ff3-480b-8c99-b867ce4a55e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:32.262619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7ebca-8ff3-480b-8c99-b867ce4a55e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:32.263451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7ebca-8ff3-480b-8c99-b867ce4a55e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:32.265184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7ebca-8ff3-480b-8c99-b867ce4a55e0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:32.278325 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1082/2183] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:32.283531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7ebca-8ff3-480b-8c99-b867ce4a55e0 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:32.296481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7ebca-8ff3-480b-8c99-b867ce4a55e0 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:32.302104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bafbf9-1f3c-46f3-9780-c0e4de6b0db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:32.302788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bafbf9-1f3c-46f3-9780-c0e4de6b0db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:32.303077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bafbf9-1f3c-46f3-9780-c0e4de6b0db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:32.303363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bafbf9-1f3c-46f3-9780-c0e4de6b0db5 admin admin] PATH_INFO: `/v3/projects/f9304713de274596b7c6484c293e64db/users/783c0c3180084599b7693877de9d2a66/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:32.321989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1102/2184] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:32.323343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bafbf9-1f3c-46f3-9780-c0e4de6b0db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9304713de274596b7c6484c293e64db, user_id=783c0c3180084599b7693877de9d2a66, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:32.326295 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bafbf9-1f3c-46f3-9780-c0e4de6b0db5 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:32.346885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029955e2-778a-494b-b865-e939b665008c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:32.347652 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-029955e2-778a-494b-b865-e939b665008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:32.378314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:32.418941 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1083/2185] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:32 2022] PUT /identity/v3/projects/f9304713de274596b7c6484c293e64db/users/783c0c3180084599b7693877de9d2a66/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:32.420758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356f42-6e14-4dfa-9808-6604c33ebb4e None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:32.421268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356f42-6e14-4dfa-9808-6604c33ebb4e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:32.421732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356f42-6e14-4dfa-9808-6604c33ebb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:32.423801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029955e2-778a-494b-b865-e939b665008c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:32.450430 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:32.455499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-09356f42-6e14-4dfa-9808-6604c33ebb4e None None] MFA Rules not processed for user `783c0c3180084599b7693877de9d2a66`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:32.457081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-09356f42-6e14-4dfa-9808-6604c33ebb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:32.516654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168baad7-c1e1-4f5b-aeb9-50c2697e2cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:32.517362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168baad7-c1e1-4f5b-aeb9-50c2697e2cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:32.519765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168baad7-c1e1-4f5b-aeb9-50c2697e2cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:32.520715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168baad7-c1e1-4f5b-aeb9-50c2697e2cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:32.580357 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1084/2186] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:32.585219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356f42-6e14-4dfa-9808-6604c33ebb4e None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:32.648936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168baad7-c1e1-4f5b-aeb9-50c2697e2cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:32.651871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168baad7-c1e1-4f5b-aeb9-50c2697e2cf0 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:32.682548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1103/2187] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:32.715880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356f42-6e14-4dfa-9808-6604c33ebb4e None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:32.733710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b077e3-71ad-4ee5-b83f-9e56c4c07217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:32.734421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b077e3-71ad-4ee5-b83f-9e56c4c07217 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:32.734844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b077e3-71ad-4ee5-b83f-9e56c4c07217 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:32.735120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b077e3-71ad-4ee5-b83f-9e56c4c07217 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:32.738288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b077e3-71ad-4ee5-b83f-9e56c4c07217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:32.753579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b077e3-71ad-4ee5-b83f-9e56c4c07217 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:32.918863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1085/2188] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:32 2022] POST /identity/v3/projects => generated 375 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:32.923243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168baad7-c1e1-4f5b-aeb9-50c2697e2cf0 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:32.940359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168baad7-c1e1-4f5b-aeb9-50c2697e2cf0 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:33.050138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf35ac8-dea6-45a7-a5a7-b6bf3b7f1de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:33.050377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf35ac8-dea6-45a7-a5a7-b6bf3b7f1de2 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:33.051539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf35ac8-dea6-45a7-a5a7-b6bf3b7f1de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:33.051777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf35ac8-dea6-45a7-a5a7-b6bf3b7f1de2 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:33.052509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf35ac8-dea6-45a7-a5a7-b6bf3b7f1de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:33.055067 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf35ac8-dea6-45a7-a5a7-b6bf3b7f1de2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:33.180732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1104/2189] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:32 2022] POST /identity/v3/users => generated 288 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:33.182134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b077e3-71ad-4ee5-b83f-9e56c4c07217 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:33.193858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b077e3-71ad-4ee5-b83f-9e56c4c07217 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:33.199687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72eea9b-8904-4907-a1df-dd05fb9d6b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:33.200495 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eea9b-8904-4907-a1df-dd05fb9d6b5f admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:33.200739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eea9b-8904-4907-a1df-dd05fb9d6b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:33.201126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72eea9b-8904-4907-a1df-dd05fb9d6b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:33.201882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72eea9b-8904-4907-a1df-dd05fb9d6b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:33.215217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72eea9b-8904-4907-a1df-dd05fb9d6b5f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:33.240379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1086/2190] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:33.243525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf35ac8-dea6-45a7-a5a7-b6bf3b7f1de2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:33.267564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf35ac8-dea6-45a7-a5a7-b6bf3b7f1de2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:33.282921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475b9d5-c24d-4197-9b32-2e8e38226111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:33.283566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475b9d5-c24d-4197-9b32-2e8e38226111 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:33.283849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475b9d5-c24d-4197-9b32-2e8e38226111 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:33.284127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475b9d5-c24d-4197-9b32-2e8e38226111 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:33.284787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475b9d5-c24d-4197-9b32-2e8e38226111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:33.286762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475b9d5-c24d-4197-9b32-2e8e38226111 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:33.298014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1105/2191] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:33.302187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72eea9b-8904-4907-a1df-dd05fb9d6b5f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:33.313461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72eea9b-8904-4907-a1df-dd05fb9d6b5f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:33.318718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8eb29-288b-4dfa-b089-5dc0f571d84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:33.320125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8eb29-288b-4dfa-b089-5dc0f571d84f admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:33.320125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8eb29-288b-4dfa-b089-5dc0f571d84f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:33.320125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8eb29-288b-4dfa-b089-5dc0f571d84f admin admin] PATH_INFO: `/v3/projects/77b7d336caec443498e30c3422d7255b/users/8ede011d94db4516a169f0ac4a4ecaa7/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:33.338897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8eb29-288b-4dfa-b089-5dc0f571d84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b7d336caec443498e30c3422d7255b, user_id=8ede011d94db4516a169f0ac4a4ecaa7, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:33.341609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8eb29-288b-4dfa-b089-5dc0f571d84f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:33.452330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1087/2192] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:33 2022] PUT /identity/v3/projects/77b7d336caec443498e30c3422d7255b/users/8ede011d94db4516a169f0ac4a4ecaa7/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:33.467120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389eece-27fd-444f-b50d-47b0139e1591 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:33.470022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389eece-27fd-444f-b50d-47b0139e1591 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:33.470022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389eece-27fd-444f-b50d-47b0139e1591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:33.482115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:33.494492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8eb29-288b-4dfa-b089-5dc0f571d84f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:33.496625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7fe8eb29-288b-4dfa-b089-5dc0f571d84f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:33.519200 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2389eece-27fd-444f-b50d-47b0139e1591 None None] MFA Rules not processed for user `8ede011d94db4516a169f0ac4a4ecaa7`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:33.520732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2389eece-27fd-444f-b50d-47b0139e1591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:33.632132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:33.651018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8eb29-288b-4dfa-b089-5dc0f571d84f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:33.657193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:33.657915 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:33.658442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:33.658914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] PATH_INFO: `/v3/users/3714c5827b2d4eeaadb7830e8b662813` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:33.660310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3714c5827b2d4eeaadb7830e8b662813)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:33.662698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:33.746791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1106/2193] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:33.759219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389eece-27fd-444f-b50d-47b0139e1591 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:33.781106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389eece-27fd-444f-b50d-47b0139e1591 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:33.783744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b4b46-3da6-41bf-84c6-b35cadc616d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:33.784737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b4b46-3da6-41bf-84c6-b35cadc616d3 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:33.785219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b4b46-3da6-41bf-84c6-b35cadc616d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:33.785796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b4b46-3da6-41bf-84c6-b35cadc616d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:33.786611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b4b46-3da6-41bf-84c6-b35cadc616d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:33.788691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b4b46-3da6-41bf-84c6-b35cadc616d3 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:33.948545 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1107/2194] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:33 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:33.952631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b4b46-3da6-41bf-84c6-b35cadc616d3 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:34.030358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3714c5827b2d4eeaadb7830e8b662813'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:34.031148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3714c5827b2d4eeaadb7830e8b662813'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:34.031830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3714c5827b2d4eeaadb7830e8b662813'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:34.048818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3714c5827b2d4eeaadb7830e8b662813'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:34.053809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b4b46-3da6-41bf-84c6-b35cadc616d3 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:34.057175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-548ab7cb-f05b-4fec-a13c-0c555a8cf4d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3714c5827b2d4eeaadb7830e8b662813'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:34.064499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da7837-da2d-4212-8507-6c36c90ee572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:34.065199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da7837-da2d-4212-8507-6c36c90ee572 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:34.065433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da7837-da2d-4212-8507-6c36c90ee572 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:34.065750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da7837-da2d-4212-8507-6c36c90ee572 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:34.066366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da7837-da2d-4212-8507-6c36c90ee572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:34.068278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da7837-da2d-4212-8507-6c36c90ee572 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:34.163863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1088/2195] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:33 2022] DELETE /identity/v3/users/3714c5827b2d4eeaadb7830e8b662813 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:34.168844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feefa42-8346-4f4b-87ef-87848784ca92 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:34.169177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feefa42-8346-4f4b-87ef-87848784ca92 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:34.169504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feefa42-8346-4f4b-87ef-87848784ca92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:34.190156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-8feefa42-8346-4f4b-87ef-87848784ca92 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:34.191613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8feefa42-8346-4f4b-87ef-87848784ca92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:34.242303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1108/2196] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:33 2022] POST /identity/v3/users => generated 295 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:34.243391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da7837-da2d-4212-8507-6c36c90ee572 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:34.248506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-66da7837-da2d-4212-8507-6c36c90ee572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:34.293889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da7837-da2d-4212-8507-6c36c90ee572 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:34.337107 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1089/2197] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:34.347186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feefa42-8346-4f4b-87ef-87848784ca92 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:34.349944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8feefa42-8346-4f4b-87ef-87848784ca92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:34.413177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5d857-2234-4233-93f2-d1575b68a110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:34.413896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5d857-2234-4233-93f2-d1575b68a110 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:34.414102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5d857-2234-4233-93f2-d1575b68a110 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:34.414369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5d857-2234-4233-93f2-d1575b68a110 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:34.415053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5d857-2234-4233-93f2-d1575b68a110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:34.421111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5d857-2234-4233-93f2-d1575b68a110 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:34.424937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:34.438606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1109/2198] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:53:34.440865 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feefa42-8346-4f4b-87ef-87848784ca92 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:34.440984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5d857-2234-4233-93f2-d1575b68a110 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:34.446636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef85da-589c-4303-82da-993aeba8b6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:34.447278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef85da-589c-4303-82da-993aeba8b6da service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:34.448149 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef85da-589c-4303-82da-993aeba8b6da service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:34.448431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef85da-589c-4303-82da-993aeba8b6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:34.452921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5d857-2234-4233-93f2-d1575b68a110 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:34.459765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a09c5c-ae9a-43ef-9144-37d146ce75e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:34.460633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a09c5c-ae9a-43ef-9144-37d146ce75e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:34.460896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a09c5c-ae9a-43ef-9144-37d146ce75e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:34.461259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a09c5c-ae9a-43ef-9144-37d146ce75e1 admin admin] PATH_INFO: `/v3/projects/1f155782e1ee44bdbf2258c5f4514dab/users/573d2462e29d4161b339d965efbf32bd/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:34.463304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef85da-589c-4303-82da-993aeba8b6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:34.465314 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef85da-589c-4303-82da-993aeba8b6da service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:34.475941 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1090/2199] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:53:34.481777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a09c5c-ae9a-43ef-9144-37d146ce75e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f155782e1ee44bdbf2258c5f4514dab, user_id=573d2462e29d4161b339d965efbf32bd, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:34.484864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a09c5c-ae9a-43ef-9144-37d146ce75e1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:34.621849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1110/2200] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:34 2022] PUT /identity/v3/projects/1f155782e1ee44bdbf2258c5f4514dab/users/573d2462e29d4161b339d965efbf32bd/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:34.625935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef85da-589c-4303-82da-993aeba8b6da service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:34.637459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef85da-589c-4303-82da-993aeba8b6da service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:34.649279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:34.707490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba693c3-3001-4acd-a630-a3f15edebdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:34.708116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba693c3-3001-4acd-a630-a3f15edebdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:34.708405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba693c3-3001-4acd-a630-a3f15edebdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:34.708682 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba693c3-3001-4acd-a630-a3f15edebdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:34.709373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba693c3-3001-4acd-a630-a3f15edebdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:34.710993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba693c3-3001-4acd-a630-a3f15edebdb9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:34.722618 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1091/2201] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:53:34.726728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a09c5c-ae9a-43ef-9144-37d146ce75e1 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:34.741362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a09c5c-ae9a-43ef-9144-37d146ce75e1 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:34.744197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac6188-ed52-4ad6-9077-164fa30c3ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:34.744943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac6188-ed52-4ad6-9077-164fa30c3ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:34.745362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac6188-ed52-4ad6-9077-164fa30c3ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:34.745644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac6188-ed52-4ad6-9077-164fa30c3ea9 admin admin] PATH_INFO: `/v3/projects/1f155782e1ee44bdbf2258c5f4514dab/users/573d2462e29d4161b339d965efbf32bd/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:34.747674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac6188-ed52-4ad6-9077-164fa30c3ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f155782e1ee44bdbf2258c5f4514dab, user_id=573d2462e29d4161b339d965efbf32bd, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:34.750926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac6188-ed52-4ad6-9077-164fa30c3ea9 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:34.763415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1111/2202] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:34 2022] PUT /identity/v3/projects/1f155782e1ee44bdbf2258c5f4514dab/users/573d2462e29d4161b339d965efbf32bd/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:34.773687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41f1c5-de42-4c43-b187-686b21163a91 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:34.773987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41f1c5-de42-4c43-b187-686b21163a91 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:34.774270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41f1c5-de42-4c43-b187-686b21163a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:34.811448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-bc41f1c5-de42-4c43-b187-686b21163a91 None None] MFA Rules not processed for user `573d2462e29d4161b339d965efbf32bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:34.816840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bc41f1c5-de42-4c43-b187-686b21163a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:34.915041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1092/2203] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:34.938317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac6188-ed52-4ad6-9077-164fa30c3ea9 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:34.940486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d6ac6188-ed52-4ad6-9077-164fa30c3ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:34.945278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41f1c5-de42-4c43-b187-686b21163a91 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:34.947940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bc41f1c5-de42-4c43-b187-686b21163a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:35.028049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac6188-ed52-4ad6-9077-164fa30c3ea9 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:35.032452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225750ce-ad48-4ea8-8ef1-dda38334c603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:35.033097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225750ce-ad48-4ea8-8ef1-dda38334c603 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:35.033377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225750ce-ad48-4ea8-8ef1-dda38334c603 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:35.035125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225750ce-ad48-4ea8-8ef1-dda38334c603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:35.036973 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-225750ce-ad48-4ea8-8ef1-dda38334c603 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:35.038888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41f1c5-de42-4c43-b187-686b21163a91 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:35.047059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:35.047710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:35.047995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:35.048273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] PATH_INFO: `/v3/projects/feb31c5abc0c4ddfb54181b5ee2a2f59` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:35.053583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feb31c5abc0c4ddfb54181b5ee2a2f59)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:35.053583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:35.111895 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225750ce-ad48-4ea8-8ef1-dda38334c603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:35.113483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225750ce-ad48-4ea8-8ef1-dda38334c603 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:35.124016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1112/2204] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:35.169600 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:35.176764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feb31c5abc0c4ddfb54181b5ee2a2f59', 'request_id': 'req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:35.187339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feb31c5abc0c4ddfb54181b5ee2a2f59', 'request_id': 'req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:35.194126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feb31c5abc0c4ddfb54181b5ee2a2f59', 'request_id': 'req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:35.266389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] The token cache is being invalidate because project feb31c5abc0c4ddfb54181b5ee2a2f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:35.266855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:35.274282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:35.274282 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1093/2205] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:34 2022] DELETE /identity/v3/projects/feb31c5abc0c4ddfb54181b5ee2a2f59 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:35.274693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225750ce-ad48-4ea8-8ef1-dda38334c603 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:35.290281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225750ce-ad48-4ea8-8ef1-dda38334c603 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:35.368185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:35.368855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:35.369166 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:35.370066 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] PATH_INFO: `/v3/users/e29b6f8c4bf54d1d8fe0b2bd1828d7b6` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:35.371921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e29b6f8c4bf54d1d8fe0b2bd1828d7b6)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:35.373520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:35.390427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:35.646950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e29b6f8c4bf54d1d8fe0b2bd1828d7b6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:35.647610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e29b6f8c4bf54d1d8fe0b2bd1828d7b6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:35.648664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e29b6f8c4bf54d1d8fe0b2bd1828d7b6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:35.658604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e29b6f8c4bf54d1d8fe0b2bd1828d7b6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:35.674459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e29b6f8c4bf54d1d8fe0b2bd1828d7b6'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:35.746831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1113/2206] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:35 2022] DELETE /identity/v3/users/e29b6f8c4bf54d1d8fe0b2bd1828d7b6 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:36.143360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:36.145110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:36.162055 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d6329-dcef-4b6b-ad3a-adbcbc675998 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:36.227211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:36.227902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:36.228119 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:36.228400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] PATH_INFO: `/v3/projects/33b4cf1e8e30452280dacb3082f957f0` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:36.229739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33b4cf1e8e30452280dacb3082f957f0)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:36.231650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:36.350235 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33b4cf1e8e30452280dacb3082f957f0', 'request_id': 'req-280ad836-4dd5-4e9e-913a-3934d1cc6693'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:36.356719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33b4cf1e8e30452280dacb3082f957f0', 'request_id': 'req-280ad836-4dd5-4e9e-913a-3934d1cc6693'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:36.362763 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33b4cf1e8e30452280dacb3082f957f0', 'request_id': 'req-280ad836-4dd5-4e9e-913a-3934d1cc6693'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:36.442026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] The token cache is being invalidate because project 33b4cf1e8e30452280dacb3082f957f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:36.442246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:36.442880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:36.444390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1094/2207] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:36 2022] DELETE /identity/v3/projects/33b4cf1e8e30452280dacb3082f957f0 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:36.449482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:36.450916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:36.464132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c8e34-5be7-41eb-9774-db74ccd1fa42 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:36.529390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:36.530033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:36.530316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:36.530591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] PATH_INFO: `/v3/users/2defe44cb47e41eebb9294a28aa7e4bc` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:36.531722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2defe44cb47e41eebb9294a28aa7e4bc)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:36.533720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:36.761668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2defe44cb47e41eebb9294a28aa7e4bc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:36.762398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2defe44cb47e41eebb9294a28aa7e4bc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:36.763073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2defe44cb47e41eebb9294a28aa7e4bc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:36.770886 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2defe44cb47e41eebb9294a28aa7e4bc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:36.785423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3acb8238-0f14-44fa-96ea-9bb21cdfb74a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2defe44cb47e41eebb9294a28aa7e4bc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:36.861715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1114/2208] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:36 2022] DELETE /identity/v3/users/2defe44cb47e41eebb9294a28aa7e4bc => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:37.341962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:37.343389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:37.357713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ad836-4dd5-4e9e-913a-3934d1cc6693 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:37.416023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:37.422631 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:37.423269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:37.423553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:37.423830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] PATH_INFO: `/v3/projects/05533c7555634b208300e8f1a380ebfb` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:37.424989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05533c7555634b208300e8f1a380ebfb)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:37.427087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:37.571342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05533c7555634b208300e8f1a380ebfb', 'request_id': 'req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:37.578469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05533c7555634b208300e8f1a380ebfb', 'request_id': 'req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:37.584054 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05533c7555634b208300e8f1a380ebfb', 'request_id': 'req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:37.659828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] The token cache is being invalidate because project 05533c7555634b208300e8f1a380ebfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:37.660115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:37.660738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-688e75f5-8fe3-4153-a47e-2abb4dbd0b75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:37.662235 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1095/2209] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:37 2022] DELETE /identity/v3/projects/05533c7555634b208300e8f1a380ebfb => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:37.667835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0040c-d464-470d-a7f1-880d07e93f31 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:37.668310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0040c-d464-470d-a7f1-880d07e93f31 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:37.668405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0040c-d464-470d-a7f1-880d07e93f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:37.688955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-bda0040c-d464-470d-a7f1-880d07e93f31 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:37.690202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-bda0040c-d464-470d-a7f1-880d07e93f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:37.699146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:37.752821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1115/2210] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:37.760752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82aff1-aa39-4d07-a078-7c5504b6fe6c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:37.761029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82aff1-aa39-4d07-a078-7c5504b6fe6c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:37.761333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82aff1-aa39-4d07-a078-7c5504b6fe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:37.781476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-3f82aff1-aa39-4d07-a078-7c5504b6fe6c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:37.782687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3f82aff1-aa39-4d07-a078-7c5504b6fe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:37.790735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1096/2211] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:37.795743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0040c-d464-470d-a7f1-880d07e93f31 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:37.805336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0040c-d464-470d-a7f1-880d07e93f31 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:37.810471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab5d50-084d-4ff2-867b-58d8d9818dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:37.811076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab5d50-084d-4ff2-867b-58d8d9818dea admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:37.811355 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab5d50-084d-4ff2-867b-58d8d9818dea admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:37.811626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab5d50-084d-4ff2-867b-58d8d9818dea admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:37.812383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab5d50-084d-4ff2-867b-58d8d9818dea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:53:37.812681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab5d50-084d-4ff2-867b-58d8d9818dea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:37.814309 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab5d50-084d-4ff2-867b-58d8d9818dea admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:37.829833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1116/2212] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:53:37.830971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82aff1-aa39-4d07-a078-7c5504b6fe6c None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:37.840949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82aff1-aa39-4d07-a078-7c5504b6fe6c None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:37.845942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2c15e-2565-4d5e-abf8-8da00a7ea2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:37.846526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2c15e-2565-4d5e-abf8-8da00a7ea2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:37.846805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2c15e-2565-4d5e-abf8-8da00a7ea2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:37.847077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2c15e-2565-4d5e-abf8-8da00a7ea2e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:37.847749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2c15e-2565-4d5e-abf8-8da00a7ea2e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:37.849751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2c15e-2565-4d5e-abf8-8da00a7ea2e3 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:37.921847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1097/2213] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:37 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:37.926183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab5d50-084d-4ff2-867b-58d8d9818dea admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:37.937759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab5d50-084d-4ff2-867b-58d8d9818dea admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:38.019055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa072e2-e07a-4d7e-b9d9-621723301588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:38.019905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa072e2-e07a-4d7e-b9d9-621723301588 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:38.020291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa072e2-e07a-4d7e-b9d9-621723301588 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:38.020636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa072e2-e07a-4d7e-b9d9-621723301588 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:38.021497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa072e2-e07a-4d7e-b9d9-621723301588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:38.026904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa072e2-e07a-4d7e-b9d9-621723301588 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:38.136538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1117/2214] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:37 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:38.140656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2c15e-2565-4d5e-abf8-8da00a7ea2e3 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:38.151835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2c15e-2565-4d5e-abf8-8da00a7ea2e3 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:38.156850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbe531-b367-4b8f-acee-f25e87e87e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:38.157527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbe531-b367-4b8f-acee-f25e87e87e43 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:38.157803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbe531-b367-4b8f-acee-f25e87e87e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:38.158076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbe531-b367-4b8f-acee-f25e87e87e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:38.158758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbe531-b367-4b8f-acee-f25e87e87e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:38.160376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbe531-b367-4b8f-acee-f25e87e87e43 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:38.176270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1098/2215] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:38.180589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa072e2-e07a-4d7e-b9d9-621723301588 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:38.194464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa072e2-e07a-4d7e-b9d9-621723301588 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:38.199724 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cef6b-7813-430a-8494-202d8927c817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:38.200830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cef6b-7813-430a-8494-202d8927c817 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:38.201172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cef6b-7813-430a-8494-202d8927c817 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:38.201452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cef6b-7813-430a-8494-202d8927c817 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:38.202101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cef6b-7813-430a-8494-202d8927c817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:38.203721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cef6b-7813-430a-8494-202d8927c817 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:38.219810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1118/2216] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:38.223911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbe531-b367-4b8f-acee-f25e87e87e43 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:38.235941 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbe531-b367-4b8f-acee-f25e87e87e43 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:38.242472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcabd55-8b41-47db-a6ad-0e7da6614b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:38.244465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcabd55-8b41-47db-a6ad-0e7da6614b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:38.244465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcabd55-8b41-47db-a6ad-0e7da6614b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:38.244754 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcabd55-8b41-47db-a6ad-0e7da6614b9d admin admin] PATH_INFO: `/v3/projects/8fcee5c6e95c467683a10d652bff9470/users/24d13f3b744542b18a056e07135d5bb7/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:38.262116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcabd55-8b41-47db-a6ad-0e7da6614b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fcee5c6e95c467683a10d652bff9470, user_id=24d13f3b744542b18a056e07135d5bb7, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:38.265879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcabd55-8b41-47db-a6ad-0e7da6614b9d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:38.360719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1099/2217] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:38 2022] PUT /identity/v3/projects/8fcee5c6e95c467683a10d652bff9470/users/24d13f3b744542b18a056e07135d5bb7/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:38.375304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bc598-7f6d-449f-888d-947467ecbce2 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:38.375587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bc598-7f6d-449f-888d-947467ecbce2 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:38.375861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bc598-7f6d-449f-888d-947467ecbce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:38.412909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f70bc598-7f6d-449f-888d-947467ecbce2 None None] MFA Rules not processed for user `24d13f3b744542b18a056e07135d5bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:38.414336 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f70bc598-7f6d-449f-888d-947467ecbce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:38.500712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1119/2218] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:38.514188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcabd55-8b41-47db-a6ad-0e7da6614b9d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:38.516306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2bcabd55-8b41-47db-a6ad-0e7da6614b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:38.598445 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcabd55-8b41-47db-a6ad-0e7da6614b9d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:38.602181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ae675-8596-4831-b729-e83c45c02c1a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d10585480f41608bc6935e282f7e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:38.602794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ae675-8596-4831-b729-e83c45c02c1a service glance] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:38.603073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ae675-8596-4831-b729-e83c45c02c1a service glance] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:38.603399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ae675-8596-4831-b729-e83c45c02c1a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:38.619045 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ae675-8596-4831-b729-e83c45c02c1a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:38.620512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ae675-8596-4831-b729-e83c45c02c1a service glance] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:38.630259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1100/2219] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:40.597615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bc598-7f6d-449f-888d-947467ecbce2 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:40.599875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f70bc598-7f6d-449f-888d-947467ecbce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:40.618216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:40.668870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bc598-7f6d-449f-888d-947467ecbce2 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:40.672475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854408a8-67d0-4ee8-ab7d-ec6e01190dea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:40.673203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854408a8-67d0-4ee8-ab7d-ec6e01190dea service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:40.673484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854408a8-67d0-4ee8-ab7d-ec6e01190dea service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:40.673807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854408a8-67d0-4ee8-ab7d-ec6e01190dea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:40.675553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-854408a8-67d0-4ee8-ab7d-ec6e01190dea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:40.784862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854408a8-67d0-4ee8-ab7d-ec6e01190dea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:40.790502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854408a8-67d0-4ee8-ab7d-ec6e01190dea service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:40.801460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1120/2220] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:42.601097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ae675-8596-4831-b729-e83c45c02c1a service glance] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:42.610878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ae675-8596-4831-b729-e83c45c02c1a service glance] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:42.618167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c9208-b835-49e0-930f-93e294c2d5c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:42.618817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c9208-b835-49e0-930f-93e294c2d5c2 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:42.619107 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c9208-b835-49e0-930f-93e294c2d5c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:42.620375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c9208-b835-49e0-930f-93e294c2d5c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:42.622438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-012c9208-b835-49e0-930f-93e294c2d5c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:42.653180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:42.691218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c9208-b835-49e0-930f-93e294c2d5c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:42.692734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c9208-b835-49e0-930f-93e294c2d5c2 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:42.705836 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1101/2221] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:43.990625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854408a8-67d0-4ee8-ab7d-ec6e01190dea service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:43.992888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-854408a8-67d0-4ee8-ab7d-ec6e01190dea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:44.081679 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854408a8-67d0-4ee8-ab7d-ec6e01190dea service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:44.086551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:44.087214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:44.088251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:44.088251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] PATH_INFO: `/v3/users/80693946241344279bad3daed358a50b` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:44.089422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80693946241344279bad3daed358a50b)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:44.091351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:44.358963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80693946241344279bad3daed358a50b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:44.359748 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80693946241344279bad3daed358a50b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:44.360437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80693946241344279bad3daed358a50b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:44.371358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80693946241344279bad3daed358a50b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:44.387014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:44.396922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80693946241344279bad3daed358a50b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:44.509177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1121/2222] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:43 2022] DELETE /identity/v3/users/80693946241344279bad3daed358a50b => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:44.514420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c9e08-b14e-4d05-ac18-cf7ff8a9c83a None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:44.514939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c9e08-b14e-4d05-ac18-cf7ff8a9c83a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:44.515210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c9e08-b14e-4d05-ac18-cf7ff8a9c83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:44.552736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-fe3c9e08-b14e-4d05-ac18-cf7ff8a9c83a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:44.554174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-fe3c9e08-b14e-4d05-ac18-cf7ff8a9c83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:44.617821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1102/2223] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:44.627698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:44.629103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:44.705430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffd814-41e0-4edc-a50e-847aca00feb4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:44.711495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ad305-4c59-4797-ae17-65704f98d37c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:44.712100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ad305-4c59-4797-ae17-65704f98d37c service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:44.712379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ad305-4c59-4797-ae17-65704f98d37c service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:44.712650 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ad305-4c59-4797-ae17-65704f98d37c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:44.722719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ad305-4c59-4797-ae17-65704f98d37c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:44.724145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ad305-4c59-4797-ae17-65704f98d37c service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:44.735455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1122/2224] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:45.182997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c9e08-b14e-4d05-ac18-cf7ff8a9c83a None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:45.184835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-fe3c9e08-b14e-4d05-ac18-cf7ff8a9c83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:45.197123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c9e08-b14e-4d05-ac18-cf7ff8a9c83a None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:45.203513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:45.204168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:45.204479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:45.204910 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] PATH_INFO: `/v3/projects/252f15925b724a579851dc4df7e9efe1` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:45.206246 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=252f15925b724a579851dc4df7e9efe1)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:45.208396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:45.364969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252f15925b724a579851dc4df7e9efe1', 'request_id': 'req-83c36925-7c28-41dd-a146-5d6b52bf5f17'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:45.373106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252f15925b724a579851dc4df7e9efe1', 'request_id': 'req-83c36925-7c28-41dd-a146-5d6b52bf5f17'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:45.379140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252f15925b724a579851dc4df7e9efe1', 'request_id': 'req-83c36925-7c28-41dd-a146-5d6b52bf5f17'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:45.495387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] The token cache is being invalidate because project 252f15925b724a579851dc4df7e9efe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:45.495741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:45.496383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c36925-7c28-41dd-a146-5d6b52bf5f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:45.497934 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1103/2225] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:45 2022] DELETE /identity/v3/projects/252f15925b724a579851dc4df7e9efe1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:45.502757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a34eb-782d-42e6-ac8e-bb35d14ce995 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:45.503185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a34eb-782d-42e6-ac8e-bb35d14ce995 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:45.503589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a34eb-782d-42e6-ac8e-bb35d14ce995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:45.525093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-200a34eb-782d-42e6-ac8e-bb35d14ce995 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:45.526694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-200a34eb-782d-42e6-ac8e-bb35d14ce995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:45.610589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1123/2226] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:45.619933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28287628-11c3-4362-bcf0-0c9cf9dc1977 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:45.620536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28287628-11c3-4362-bcf0-0c9cf9dc1977 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:45.623096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28287628-11c3-4362-bcf0-0c9cf9dc1977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:45.648134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-28287628-11c3-4362-bcf0-0c9cf9dc1977 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:45.649321 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-28287628-11c3-4362-bcf0-0c9cf9dc1977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:45.658327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1104/2227] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:45.662819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a34eb-782d-42e6-ac8e-bb35d14ce995 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:45.672830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a34eb-782d-42e6-ac8e-bb35d14ce995 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:45.678549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35e1d7-1e21-44ff-8547-0f165352c78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:45.679242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35e1d7-1e21-44ff-8547-0f165352c78f admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:45.683573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35e1d7-1e21-44ff-8547-0f165352c78f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:45.683573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35e1d7-1e21-44ff-8547-0f165352c78f admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:45.683573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35e1d7-1e21-44ff-8547-0f165352c78f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:53:45.683573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35e1d7-1e21-44ff-8547-0f165352c78f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:45.683573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35e1d7-1e21-44ff-8547-0f165352c78f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:45.697092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1124/2228] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:53:45.701064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28287628-11c3-4362-bcf0-0c9cf9dc1977 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:45.711567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28287628-11c3-4362-bcf0-0c9cf9dc1977 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:45.717343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0142db-7e0c-41f5-89e4-4879a8cbaca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:45.718070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0142db-7e0c-41f5-89e4-4879a8cbaca1 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:45.718389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0142db-7e0c-41f5-89e4-4879a8cbaca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:45.718907 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0142db-7e0c-41f5-89e4-4879a8cbaca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:45.719562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0142db-7e0c-41f5-89e4-4879a8cbaca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:45.721851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0142db-7e0c-41f5-89e4-4879a8cbaca1 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:45.832059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1105/2229] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:45 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:45.833541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35e1d7-1e21-44ff-8547-0f165352c78f admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:45.846972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35e1d7-1e21-44ff-8547-0f165352c78f admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:45.934255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78c42f-561e-448f-ae7d-d69b90abf592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:45.934917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78c42f-561e-448f-ae7d-d69b90abf592 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:45.935198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78c42f-561e-448f-ae7d-d69b90abf592 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:45.935535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78c42f-561e-448f-ae7d-d69b90abf592 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:45.936211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78c42f-561e-448f-ae7d-d69b90abf592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:45.938276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78c42f-561e-448f-ae7d-d69b90abf592 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:46.043045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1125/2230] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:45 2022] POST /identity/v3/users => generated 288 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:46.045014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0142db-7e0c-41f5-89e4-4879a8cbaca1 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:46.062565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0142db-7e0c-41f5-89e4-4879a8cbaca1 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:46.078279 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5ee4e-04dc-4a89-b3b6-4b4c551fe1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:46.078928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5ee4e-04dc-4a89-b3b6-4b4c551fe1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:46.079260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5ee4e-04dc-4a89-b3b6-4b4c551fe1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:46.079533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5ee4e-04dc-4a89-b3b6-4b4c551fe1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:46.080199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5ee4e-04dc-4a89-b3b6-4b4c551fe1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:46.081905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5ee4e-04dc-4a89-b3b6-4b4c551fe1d6 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:46.100096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1106/2231] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:46.105966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78c42f-561e-448f-ae7d-d69b90abf592 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:46.112319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:46.118846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78c42f-561e-448f-ae7d-d69b90abf592 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:46.124551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797f47c-07f4-4993-be34-cce7f8c2cc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:46.125201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797f47c-07f4-4993-be34-cce7f8c2cc66 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:46.125559 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797f47c-07f4-4993-be34-cce7f8c2cc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:46.125823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797f47c-07f4-4993-be34-cce7f8c2cc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:46.126471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797f47c-07f4-4993-be34-cce7f8c2cc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:46.128150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797f47c-07f4-4993-be34-cce7f8c2cc66 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:46.141267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1126/2232] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:46.145872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5ee4e-04dc-4a89-b3b6-4b4c551fe1d6 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:46.153226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:46.165673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5ee4e-04dc-4a89-b3b6-4b4c551fe1d6 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:46.173022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e32a7-345d-4649-a95c-533e98b5971c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:46.174069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e32a7-345d-4649-a95c-533e98b5971c admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:46.174509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e32a7-345d-4649-a95c-533e98b5971c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:46.175201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e32a7-345d-4649-a95c-533e98b5971c admin admin] PATH_INFO: `/v3/projects/c7048c8eeb5e45cf9dbe72a59f9c642a/users/9bd98c869fb3418bb6b0e0f186aa8836/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:46.192909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e32a7-345d-4649-a95c-533e98b5971c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7048c8eeb5e45cf9dbe72a59f9c642a, user_id=9bd98c869fb3418bb6b0e0f186aa8836, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:46.195633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e32a7-345d-4649-a95c-533e98b5971c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:46.291492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1107/2233] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:46 2022] PUT /identity/v3/projects/c7048c8eeb5e45cf9dbe72a59f9c642a/users/9bd98c869fb3418bb6b0e0f186aa8836/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:46.302373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860baf0-254f-4a09-8ef5-49317d2c26e8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:46.302663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860baf0-254f-4a09-8ef5-49317d2c26e8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:46.302938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860baf0-254f-4a09-8ef5-49317d2c26e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:46.340759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f860baf0-254f-4a09-8ef5-49317d2c26e8 None None] MFA Rules not processed for user `9bd98c869fb3418bb6b0e0f186aa8836`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:46.342214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f860baf0-254f-4a09-8ef5-49317d2c26e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:46.454579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1127/2234] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:46.456288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e32a7-345d-4649-a95c-533e98b5971c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:46.529148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e32a7-345d-4649-a95c-533e98b5971c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:46.534009 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043b75c-5448-4004-9e05-eab239c06306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:46.534804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043b75c-5448-4004-9e05-eab239c06306 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:46.535197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043b75c-5448-4004-9e05-eab239c06306 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:46.535473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043b75c-5448-4004-9e05-eab239c06306 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:46.536204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043b75c-5448-4004-9e05-eab239c06306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:46.538414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043b75c-5448-4004-9e05-eab239c06306 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:46.624220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1108/2235] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:46 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:46.630827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860baf0-254f-4a09-8ef5-49317d2c26e8 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:46.642579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860baf0-254f-4a09-8ef5-49317d2c26e8 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:46.705594 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35e017-dd9b-439f-8c60-102c1dd54cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:46.706130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35e017-dd9b-439f-8c60-102c1dd54cfb admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:46.706412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35e017-dd9b-439f-8c60-102c1dd54cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:46.706686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35e017-dd9b-439f-8c60-102c1dd54cfb admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:46.707360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35e017-dd9b-439f-8c60-102c1dd54cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:46.709366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35e017-dd9b-439f-8c60-102c1dd54cfb admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:46.854585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1128/2236] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:46 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:46.858545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043b75c-5448-4004-9e05-eab239c06306 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:46.868970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043b75c-5448-4004-9e05-eab239c06306 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:46.874302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e41b4-6bf9-4d89-81a0-e350452a9d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:46.874905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e41b4-6bf9-4d89-81a0-e350452a9d40 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:46.875184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e41b4-6bf9-4d89-81a0-e350452a9d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:46.875460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e41b4-6bf9-4d89-81a0-e350452a9d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:46.876107 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e41b4-6bf9-4d89-81a0-e350452a9d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:46.877752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e41b4-6bf9-4d89-81a0-e350452a9d40 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:46.889213 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1109/2237] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:46.893521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35e017-dd9b-439f-8c60-102c1dd54cfb admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:46.904215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35e017-dd9b-439f-8c60-102c1dd54cfb admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:46.908872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e5c6a-a5f1-45d8-8e42-caaf027df5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:46.909709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e5c6a-a5f1-45d8-8e42-caaf027df5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:46.909992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e5c6a-a5f1-45d8-8e42-caaf027df5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:46.910265 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e5c6a-a5f1-45d8-8e42-caaf027df5fa admin admin] PATH_INFO: `/v3/projects/9961103df0134a7ea78c9074d186e02b/users/ceb32f50bf4b4d13ab4706d0515d2ba0/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:46.925742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e5c6a-a5f1-45d8-8e42-caaf027df5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9961103df0134a7ea78c9074d186e02b, user_id=ceb32f50bf4b4d13ab4706d0515d2ba0, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:46.928607 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e5c6a-a5f1-45d8-8e42-caaf027df5fa admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:47.049905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1129/2238] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:46 2022] PUT /identity/v3/projects/9961103df0134a7ea78c9074d186e02b/users/ceb32f50bf4b4d13ab4706d0515d2ba0/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:47.054062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e41b4-6bf9-4d89-81a0-e350452a9d40 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:47.064866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e41b4-6bf9-4d89-81a0-e350452a9d40 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:47.135442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c31ff-a6f0-4c37-9667-15534c7f0810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:47.136231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c31ff-a6f0-4c37-9667-15534c7f0810 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:47.136582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c31ff-a6f0-4c37-9667-15534c7f0810 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:47.136927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c31ff-a6f0-4c37-9667-15534c7f0810 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:47.137676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c31ff-a6f0-4c37-9667-15534c7f0810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:47.139382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c31ff-a6f0-4c37-9667-15534c7f0810 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:47.150250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1110/2239] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:47.155148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e5c6a-a5f1-45d8-8e42-caaf027df5fa admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:47.172080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e5c6a-a5f1-45d8-8e42-caaf027df5fa admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:47.176915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a3dc9-bf62-4b23-973d-0829ba18970b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:47.177643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a3dc9-bf62-4b23-973d-0829ba18970b admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:47.182704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a3dc9-bf62-4b23-973d-0829ba18970b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:47.182704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a3dc9-bf62-4b23-973d-0829ba18970b admin admin] PATH_INFO: `/v3/projects/9961103df0134a7ea78c9074d186e02b/users/ceb32f50bf4b4d13ab4706d0515d2ba0/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:47.182704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a3dc9-bf62-4b23-973d-0829ba18970b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9961103df0134a7ea78c9074d186e02b, user_id=ceb32f50bf4b4d13ab4706d0515d2ba0, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:47.183145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a3dc9-bf62-4b23-973d-0829ba18970b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:47.194130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1130/2240] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:47 2022] PUT /identity/v3/projects/9961103df0134a7ea78c9074d186e02b/users/ceb32f50bf4b4d13ab4706d0515d2ba0/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:47.206141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff06944-5d5c-4915-97ae-54b63ee78efd None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:47.206456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff06944-5d5c-4915-97ae-54b63ee78efd None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:47.206775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff06944-5d5c-4915-97ae-54b63ee78efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:47.261372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-6ff06944-5d5c-4915-97ae-54b63ee78efd None None] MFA Rules not processed for user `ceb32f50bf4b4d13ab4706d0515d2ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:47.287659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6ff06944-5d5c-4915-97ae-54b63ee78efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:47.321672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:47.348580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1111/2241] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:47.360379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a3dc9-bf62-4b23-973d-0829ba18970b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:47.362409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-989a3dc9-bf62-4b23-973d-0829ba18970b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:47.404603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:47.545216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a3dc9-bf62-4b23-973d-0829ba18970b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:47.545216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736661-c660-419a-9449-7f3d9f300cc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:47.545216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736661-c660-419a-9449-7f3d9f300cc8 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:47.545216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736661-c660-419a-9449-7f3d9f300cc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:47.547561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736661-c660-419a-9449-7f3d9f300cc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:47.547561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b736661-c660-419a-9449-7f3d9f300cc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:47.547561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b736661-c660-419a-9449-7f3d9f300cc8 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:47.547561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1131/2242] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:47.550127 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff06944-5d5c-4915-97ae-54b63ee78efd None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:47.551104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6ff06944-5d5c-4915-97ae-54b63ee78efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:47.651221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff06944-5d5c-4915-97ae-54b63ee78efd None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:47.658130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:47.658766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:47.659056 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:47.659472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] PATH_INFO: `/v3/users/cc1586d8d1e54235ab4e54daa3c6c249` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:47.661375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc1586d8d1e54235ab4e54daa3c6c249)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:47.664330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:47.748542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736661-c660-419a-9449-7f3d9f300cc8 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:47.750638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1b736661-c660-419a-9449-7f3d9f300cc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:47.769193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736661-c660-419a-9449-7f3d9f300cc8 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:47.776516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:47.777922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:47.778280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:47.778931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] PATH_INFO: `/v3/users/783c0c3180084599b7693877de9d2a66` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:47.781014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=783c0c3180084599b7693877de9d2a66)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:47.791787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:47.913532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc1586d8d1e54235ab4e54daa3c6c249'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:47.914296 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc1586d8d1e54235ab4e54daa3c6c249'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:47.915049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc1586d8d1e54235ab4e54daa3c6c249'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:47.928624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc1586d8d1e54235ab4e54daa3c6c249'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:47.937253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a6a7a6ad-e211-4fa4-9d8a-f346839bcb50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc1586d8d1e54235ab4e54daa3c6c249'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:47.985913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '783c0c3180084599b7693877de9d2a66'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:47.986980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '783c0c3180084599b7693877de9d2a66'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:47.987615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '783c0c3180084599b7693877de9d2a66'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:48.002703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1112/2243] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:47 2022] DELETE /identity/v3/users/cc1586d8d1e54235ab4e54daa3c6c249 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:48.006259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3d687-d65d-4f76-b75b-3e9034fb7121 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:48.006567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3d687-d65d-4f76-b75b-3e9034fb7121 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:48.006843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3d687-d65d-4f76-b75b-3e9034fb7121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:48.015498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '783c0c3180084599b7693877de9d2a66'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:48.029085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e6e3d687-d65d-4f76-b75b-3e9034fb7121 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:48.033193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e6e3d687-d65d-4f76-b75b-3e9034fb7121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:48.034020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-76ced048-2fa9-4d0b-a3a0-670d3c8bacc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '783c0c3180084599b7693877de9d2a66'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:48.081521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:48.096481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1113/2244] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:48.107309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3d687-d65d-4f76-b75b-3e9034fb7121 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:48.108837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e6e3d687-d65d-4f76-b75b-3e9034fb7121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:48.135783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1132/2245] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:47 2022] DELETE /identity/v3/users/783c0c3180084599b7693877de9d2a66 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:48.140143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f7e72-edda-419c-9b69-f0ec04e27eeb None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:48.140437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f7e72-edda-419c-9b69-f0ec04e27eeb None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:48.140719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f7e72-edda-419c-9b69-f0ec04e27eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:48.161244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-123f7e72-edda-419c-9b69-f0ec04e27eeb None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:48.162396 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-123f7e72-edda-419c-9b69-f0ec04e27eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:48.198936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3d687-d65d-4f76-b75b-3e9034fb7121 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:48.229337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:48.239815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1133/2246] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:48.254945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f7e72-edda-419c-9b69-f0ec04e27eeb None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:48.276791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a2f60-b9c0-4bc0-a51e-9bcfbc1c096e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:48.277557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a2f60-b9c0-4bc0-a51e-9bcfbc1c096e service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:48.277953 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a2f60-b9c0-4bc0-a51e-9bcfbc1c096e service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:48.278313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a2f60-b9c0-4bc0-a51e-9bcfbc1c096e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:48.294303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a2f60-b9c0-4bc0-a51e-9bcfbc1c096e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:48.295781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a2f60-b9c0-4bc0-a51e-9bcfbc1c096e service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:48.306543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1114/2247] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:53:48.334637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f7e72-edda-419c-9b69-f0ec04e27eeb None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:48.340279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8f952-9619-48c0-a93a-a19bd84d58da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:48.341027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f952-9619-48c0-a93a-a19bd84d58da service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:48.341422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f952-9619-48c0-a93a-a19bd84d58da service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:48.341666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8f952-9619-48c0-a93a-a19bd84d58da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:48.361080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8f952-9619-48c0-a93a-a19bd84d58da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:48.362679 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8f952-9619-48c0-a93a-a19bd84d58da service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:48.384768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1134/2248] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:53:48.993629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a2f60-b9c0-4bc0-a51e-9bcfbc1c096e service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:48.995710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4e0a2f60-b9c0-4bc0-a51e-9bcfbc1c096e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:49.019806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a2f60-b9c0-4bc0-a51e-9bcfbc1c096e service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:49.024547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:49.025623 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.026096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.026384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] PATH_INFO: `/v3/projects/135f8764fafd4a39bee1570486a1d92f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.028160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=135f8764fafd4a39bee1570486a1d92f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:49.030138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:49.144911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '135f8764fafd4a39bee1570486a1d92f', 'request_id': 'req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.152899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '135f8764fafd4a39bee1570486a1d92f', 'request_id': 'req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.158926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '135f8764fafd4a39bee1570486a1d92f', 'request_id': 'req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.199455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8f952-9619-48c0-a93a-a19bd84d58da service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:49.207143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-34b8f952-9619-48c0-a93a-a19bd84d58da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:49.227288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8f952-9619-48c0-a93a-a19bd84d58da service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:49.235558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] The token cache is being invalidate because project 135f8764fafd4a39bee1570486a1d92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:49.235558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.235558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5d4d950a-98dc-4997-8d4c-afc6dd8a9597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.237922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1115/2249] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:48 2022] DELETE /identity/v3/projects/135f8764fafd4a39bee1570486a1d92f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:49.243094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024ed6-7cf3-4e19-9bde-ad1e82f454fb None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.243390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024ed6-7cf3-4e19-9bde-ad1e82f454fb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.243669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024ed6-7cf3-4e19-9bde-ad1e82f454fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.271696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-cc024ed6-7cf3-4e19-9bde-ad1e82f454fb None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:49.272829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cc024ed6-7cf3-4e19-9bde-ad1e82f454fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:49.309870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:49.312403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:49.314391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.315930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.315930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] PATH_INFO: `/v3/projects/f9304713de274596b7c6484c293e64db` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.316373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9304713de274596b7c6484c293e64db)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:49.318631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:49.350772 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1116/2250] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:49.359224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074905f0-9bc4-4370-b430-410634fe0880 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.359565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074905f0-9bc4-4370-b430-410634fe0880 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.359849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074905f0-9bc4-4370-b430-410634fe0880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.394614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-074905f0-9bc4-4370-b430-410634fe0880 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:49.395910 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-074905f0-9bc4-4370-b430-410634fe0880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:49.405926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1117/2251] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:49.417110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074905f0-9bc4-4370-b430-410634fe0880 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:49.426880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074905f0-9bc4-4370-b430-410634fe0880 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:49.433825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7988df-838b-4967-9392-8a5bc73fc8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:49.439039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7988df-838b-4967-9392-8a5bc73fc8fc admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.439039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7988df-838b-4967-9392-8a5bc73fc8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.439039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7988df-838b-4967-9392-8a5bc73fc8fc admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.439039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7988df-838b-4967-9392-8a5bc73fc8fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:53:49.439039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7988df-838b-4967-9392-8a5bc73fc8fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:49.439039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7988df-838b-4967-9392-8a5bc73fc8fc admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:49.453300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1118/2252] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:53:49.459119 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7988df-838b-4967-9392-8a5bc73fc8fc admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:49.472081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7988df-838b-4967-9392-8a5bc73fc8fc admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:49.477672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e471c78-aa66-4519-b7ec-d2a87b1f43ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:49.478278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e471c78-aa66-4519-b7ec-d2a87b1f43ba admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.479151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e471c78-aa66-4519-b7ec-d2a87b1f43ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.479151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e471c78-aa66-4519-b7ec-d2a87b1f43ba admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.479850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e471c78-aa66-4519-b7ec-d2a87b1f43ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:49.482166 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e471c78-aa66-4519-b7ec-d2a87b1f43ba admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:49.493984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9304713de274596b7c6484c293e64db', 'request_id': 'req-00fc14d9-ecc1-46a2-b358-acfbee0825c4'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.501070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9304713de274596b7c6484c293e64db', 'request_id': 'req-00fc14d9-ecc1-46a2-b358-acfbee0825c4'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.567260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1119/2253] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:49 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:49.568293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e471c78-aa66-4519-b7ec-d2a87b1f43ba admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:49.579435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9304713de274596b7c6484c293e64db', 'request_id': 'req-00fc14d9-ecc1-46a2-b358-acfbee0825c4'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.591912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] The token cache is being invalidate because project f9304713de274596b7c6484c293e64db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:49.592663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.593016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.597477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1135/2254] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:49 2022] DELETE /identity/v3/projects/f9304713de274596b7c6484c293e64db => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:49.598510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:49.599878 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:49.640487 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e471c78-aa66-4519-b7ec-d2a87b1f43ba admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:49.644087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:49.645416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded5a8b-2938-4816-9733-e1a2eccfeedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:49.646039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5a8b-2938-4816-9733-e1a2eccfeedd admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.646375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5a8b-2938-4816-9733-e1a2eccfeedd admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.646652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5a8b-2938-4816-9733-e1a2eccfeedd admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.647291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded5a8b-2938-4816-9733-e1a2eccfeedd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:49.649686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded5a8b-2938-4816-9733-e1a2eccfeedd admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:49.699871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc14d9-ecc1-46a2-b358-acfbee0825c4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:49.705131 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:49.705970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.706381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.706731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] PATH_INFO: `/v3/users/8ede011d94db4516a169f0ac4a4ecaa7` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.708010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ede011d94db4516a169f0ac4a4ecaa7)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:49.712721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:49.758775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1120/2255] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:49 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:49.763061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded5a8b-2938-4816-9733-e1a2eccfeedd admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:49.765931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cded5a8b-2938-4816-9733-e1a2eccfeedd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:49.780289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded5a8b-2938-4816-9733-e1a2eccfeedd admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:49.788292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd44002-8ec6-491f-b9c2-68113f343729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:49.790492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd44002-8ec6-491f-b9c2-68113f343729 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.790850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd44002-8ec6-491f-b9c2-68113f343729 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.791277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd44002-8ec6-491f-b9c2-68113f343729 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.792026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd44002-8ec6-491f-b9c2-68113f343729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:49.793833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd44002-8ec6-491f-b9c2-68113f343729 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:49.808315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1121/2256] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:49.812154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd44002-8ec6-491f-b9c2-68113f343729 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:49.823448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd44002-8ec6-491f-b9c2-68113f343729 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:49.829350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d099f-e737-43c0-951d-0260c7fc8f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:49.829960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d099f-e737-43c0-951d-0260c7fc8f41 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.830239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d099f-e737-43c0-951d-0260c7fc8f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.830511 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d099f-e737-43c0-951d-0260c7fc8f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.831159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d099f-e737-43c0-951d-0260c7fc8f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:49.832767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d099f-e737-43c0-951d-0260c7fc8f41 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:49.845602 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1122/2257] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:53:49.850384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d099f-e737-43c0-951d-0260c7fc8f41 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:49.868846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d099f-e737-43c0-951d-0260c7fc8f41 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:49.874784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a80847-2a29-4a3a-afd8-6738693fe1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:49.875660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a80847-2a29-4a3a-afd8-6738693fe1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.875946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a80847-2a29-4a3a-afd8-6738693fe1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.877211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a80847-2a29-4a3a-afd8-6738693fe1ea admin admin] PATH_INFO: `/v3/projects/0aadff73d43243d794a8552eac667e4f/users/24f1de8878174b36ae83b2afdc960b84/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:49.895789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a80847-2a29-4a3a-afd8-6738693fe1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aadff73d43243d794a8552eac667e4f, user_id=24f1de8878174b36ae83b2afdc960b84, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:49.898716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a80847-2a29-4a3a-afd8-6738693fe1ea admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:49.933987 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ede011d94db4516a169f0ac4a4ecaa7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.934697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ede011d94db4516a169f0ac4a4ecaa7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.935405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ede011d94db4516a169f0ac4a4ecaa7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.942636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ede011d94db4516a169f0ac4a4ecaa7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.955175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ede011d94db4516a169f0ac4a4ecaa7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:49.969896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1123/2258] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:49 2022] PUT /identity/v3/projects/0aadff73d43243d794a8552eac667e4f/users/24f1de8878174b36ae83b2afdc960b84/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:49.982145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6c40f-f33a-43df-9e9f-ce689779b840 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:49.982447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6c40f-f33a-43df-9e9f-ce689779b840 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:49.982721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6c40f-f33a-43df-9e9f-ce689779b840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:50.010891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f9a6c40f-f33a-43df-9e9f-ce689779b840 None None] MFA Rules not processed for user `24f1de8878174b36ae83b2afdc960b84`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:50.012176 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f9a6c40f-f33a-43df-9e9f-ce689779b840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:50.033090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1136/2259] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:49 2022] DELETE /identity/v3/users/8ede011d94db4516a169f0ac4a4ecaa7 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:50.042809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:50.094083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1124/2260] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:50.106387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:50.107945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:50.168813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca139b-c9e4-4167-bdb2-c3ab1998725a admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:50.172680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f54a5-3283-4cf0-b321-1b61c4d73de0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:50.173317 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f54a5-3283-4cf0-b321-1b61c4d73de0 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:50.173600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f54a5-3283-4cf0-b321-1b61c4d73de0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:50.173876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f54a5-3283-4cf0-b321-1b61c4d73de0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:50.236293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f54a5-3283-4cf0-b321-1b61c4d73de0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:50.238022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f54a5-3283-4cf0-b321-1b61c4d73de0 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:50.247362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1137/2261] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:50.529998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6c40f-f33a-43df-9e9f-ce689779b840 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:50.531806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f9a6c40f-f33a-43df-9e9f-ce689779b840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:50.610297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6c40f-f33a-43df-9e9f-ce689779b840 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:50.687626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:50.690181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:50.690181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:50.690181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] PATH_INFO: `/v3/projects/77b7d336caec443498e30c3422d7255b` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:50.690181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b7d336caec443498e30c3422d7255b)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:50.691792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:50.761794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f54a5-3283-4cf0-b321-1b61c4d73de0 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:50.763315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-572f54a5-3283-4cf0-b321-1b61c4d73de0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:50.777719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f54a5-3283-4cf0-b321-1b61c4d73de0 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:50.782284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:50.782911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:50.783194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:50.783495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] PATH_INFO: `/v3/users/9bd98c869fb3418bb6b0e0f186aa8836` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:50.784607 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd98c869fb3418bb6b0e0f186aa8836)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:50.786555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:50.788127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:50.825989 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b7d336caec443498e30c3422d7255b', 'request_id': 'req-4292b3f5-a4e2-4eb2-946c-32e372080eac'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:50.833979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b7d336caec443498e30c3422d7255b', 'request_id': 'req-4292b3f5-a4e2-4eb2-946c-32e372080eac'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:50.850061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b7d336caec443498e30c3422d7255b', 'request_id': 'req-4292b3f5-a4e2-4eb2-946c-32e372080eac'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:50.932022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] The token cache is being invalidate because project 77b7d336caec443498e30c3422d7255b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:50.932022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:50.932022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:50.934685 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1125/2262] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:50 2022] DELETE /identity/v3/projects/77b7d336caec443498e30c3422d7255b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:50.938298 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:50.941357 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:50.952806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:51.046979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd98c869fb3418bb6b0e0f186aa8836'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:51.052574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd98c869fb3418bb6b0e0f186aa8836'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:51.052574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd98c869fb3418bb6b0e0f186aa8836'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:51.061256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292b3f5-a4e2-4eb2-946c-32e372080eac admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:51.061774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd98c869fb3418bb6b0e0f186aa8836'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:51.082011 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:51.082695 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:51.082983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:51.083267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] PATH_INFO: `/v3/users/573d2462e29d4161b339d965efbf32bd` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:51.084522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=573d2462e29d4161b339d965efbf32bd)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:51.084952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-208ad4bb-75d0-4f1c-9629-1f47c39a19b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd98c869fb3418bb6b0e0f186aa8836'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:51.086543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:51.193197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1138/2263] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:50 2022] DELETE /identity/v3/users/9bd98c869fb3418bb6b0e0f186aa8836 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:51.197874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0fce8-f7b6-4b36-ae34-5d1f3933ba64 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:51.198164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0fce8-f7b6-4b36-ae34-5d1f3933ba64 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:51.198654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0fce8-f7b6-4b36-ae34-5d1f3933ba64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:51.242730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c4b0fce8-f7b6-4b36-ae34-5d1f3933ba64 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:51.245173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c4b0fce8-f7b6-4b36-ae34-5d1f3933ba64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:51.305365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1139/2264] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:51.316014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0fce8-f7b6-4b36-ae34-5d1f3933ba64 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:51.317291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573d2462e29d4161b339d965efbf32bd'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:51.317449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c4b0fce8-f7b6-4b36-ae34-5d1f3933ba64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:51.318102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573d2462e29d4161b339d965efbf32bd'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:51.318722 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573d2462e29d4161b339d965efbf32bd'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:51.331182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573d2462e29d4161b339d965efbf32bd'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:51.338937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573d2462e29d4161b339d965efbf32bd'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:51.378039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0fce8-f7b6-4b36-ae34-5d1f3933ba64 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:51.381971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1af06c-7ccd-4175-a60a-1c755369344f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:51.382700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1af06c-7ccd-4175-a60a-1c755369344f service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:51.382977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1af06c-7ccd-4175-a60a-1c755369344f service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:51.383252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1af06c-7ccd-4175-a60a-1c755369344f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:51.385289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9d1af06c-7ccd-4175-a60a-1c755369344f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:51.396802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1af06c-7ccd-4175-a60a-1c755369344f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:51.399233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1af06c-7ccd-4175-a60a-1c755369344f service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:51.410946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1140/2265] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:53:51.434955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1126/2266] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:50 2022] DELETE /identity/v3/users/573d2462e29d4161b339d965efbf32bd => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:51.961430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1af06c-7ccd-4175-a60a-1c755369344f service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:51.967124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9d1af06c-7ccd-4175-a60a-1c755369344f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:51.994777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1af06c-7ccd-4175-a60a-1c755369344f service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:52.002288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:52.004387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:52.045675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:52.105936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ce893-1ea5-42a9-a4b1-48e199f77151 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:52.112671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:52.114506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:52.114506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:52.114506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] PATH_INFO: `/v3/projects/c7048c8eeb5e45cf9dbe72a59f9c642a` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:52.115075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7048c8eeb5e45cf9dbe72a59f9c642a)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:52.117342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:52.163887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:52.163887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:52.163887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:52.163887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] PATH_INFO: `/v3/projects/1f155782e1ee44bdbf2258c5f4514dab` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:52.165560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f155782e1ee44bdbf2258c5f4514dab)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:52.169625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:52.225402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:52.259016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7048c8eeb5e45cf9dbe72a59f9c642a', 'request_id': 'req-a690dd49-9f96-4e65-84ec-7a441ac46306'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.270392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7048c8eeb5e45cf9dbe72a59f9c642a', 'request_id': 'req-a690dd49-9f96-4e65-84ec-7a441ac46306'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.282269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7048c8eeb5e45cf9dbe72a59f9c642a', 'request_id': 'req-a690dd49-9f96-4e65-84ec-7a441ac46306'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.314275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f155782e1ee44bdbf2258c5f4514dab', 'request_id': 'req-61d92202-71ca-429e-a4d7-0f3240c0dffd'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.328629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f155782e1ee44bdbf2258c5f4514dab', 'request_id': 'req-61d92202-71ca-429e-a4d7-0f3240c0dffd'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.375125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] The token cache is being invalidate because project c7048c8eeb5e45cf9dbe72a59f9c642a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:52.382894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.382894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.382894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1127/2267] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:52 2022] DELETE /identity/v3/projects/c7048c8eeb5e45cf9dbe72a59f9c642a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:52.385159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:52.387293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:52.455636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690dd49-9f96-4e65-84ec-7a441ac46306 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:52.522367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:52.523018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:52.523356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:52.523572 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] PATH_INFO: `/v3/users/ceb32f50bf4b4d13ab4706d0515d2ba0` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:52.524699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb32f50bf4b4d13ab4706d0515d2ba0)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:52.527068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:52.529160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f155782e1ee44bdbf2258c5f4514dab', 'request_id': 'req-61d92202-71ca-429e-a4d7-0f3240c0dffd'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.549720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] The token cache is being invalidate because project 1f155782e1ee44bdbf2258c5f4514dab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:52.550507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.551504 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-61d92202-71ca-429e-a4d7-0f3240c0dffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.553548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1141/2268] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:51 2022] DELETE /identity/v3/projects/1f155782e1ee44bdbf2258c5f4514dab => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:52.560274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba076a8e-1ab1-4ee1-b19f-2e33cdd1743a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:52.560933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba076a8e-1ab1-4ee1-b19f-2e33cdd1743a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:52.561392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba076a8e-1ab1-4ee1-b19f-2e33cdd1743a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:52.583621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-ba076a8e-1ab1-4ee1-b19f-2e33cdd1743a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:52.584848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ba076a8e-1ab1-4ee1-b19f-2e33cdd1743a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:52.593034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1142/2269] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:52.602928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5a1ce-96c6-4e6a-ac46-f82be4c97c74 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:52.603218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5a1ce-96c6-4e6a-ac46-f82be4c97c74 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:52.604075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5a1ce-96c6-4e6a-ac46-f82be4c97c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:52.625671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b2f5a1ce-96c6-4e6a-ac46-f82be4c97c74 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:52.626875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b2f5a1ce-96c6-4e6a-ac46-f82be4c97c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:52.635527 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1143/2270] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:52.639526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5a1ce-96c6-4e6a-ac46-f82be4c97c74 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:52.649137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5a1ce-96c6-4e6a-ac46-f82be4c97c74 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:52.654112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3dfa03-89b1-4da6-ade4-194ea55d249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:52.654687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dfa03-89b1-4da6-ade4-194ea55d249f admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:52.654963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dfa03-89b1-4da6-ade4-194ea55d249f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:52.655229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dfa03-89b1-4da6-ade4-194ea55d249f admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:52.655889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dfa03-89b1-4da6-ade4-194ea55d249f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:53:52.656186 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dfa03-89b1-4da6-ade4-194ea55d249f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:52.657850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dfa03-89b1-4da6-ade4-194ea55d249f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:52.676744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1144/2271] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:53:52.682709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3dfa03-89b1-4da6-ade4-194ea55d249f admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:52.693943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3dfa03-89b1-4da6-ade4-194ea55d249f admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:52.699583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e7021-e7ed-47af-8bbb-58285bd2c3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:52.700215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e7021-e7ed-47af-8bbb-58285bd2c3a1 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:52.700495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e7021-e7ed-47af-8bbb-58285bd2c3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:52.700875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e7021-e7ed-47af-8bbb-58285bd2c3a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:52.701692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e7021-e7ed-47af-8bbb-58285bd2c3a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:52.703630 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e7021-e7ed-47af-8bbb-58285bd2c3a1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:52.800043 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1145/2272] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:52 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:52.803689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb32f50bf4b4d13ab4706d0515d2ba0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.804592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb32f50bf4b4d13ab4706d0515d2ba0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.804920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e7021-e7ed-47af-8bbb-58285bd2c3a1 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:52.805540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb32f50bf4b4d13ab4706d0515d2ba0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.818332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2e4e7021-e7ed-47af-8bbb-58285bd2c3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:52.819978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb32f50bf4b4d13ab4706d0515d2ba0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.821202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:52.828052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb32f50bf4b4d13ab4706d0515d2ba0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:52.901565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1128/2273] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:53:52 2022] DELETE /identity/v3/users/ceb32f50bf4b4d13ab4706d0515d2ba0 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:52.906804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e7021-e7ed-47af-8bbb-58285bd2c3a1 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:52.918728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:52.997998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab52356-6a24-4176-8540-baf59057aa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:53.000320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab52356-6a24-4176-8540-baf59057aa48 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:53.000320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab52356-6a24-4176-8540-baf59057aa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:53.000320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab52356-6a24-4176-8540-baf59057aa48 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:53.000320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab52356-6a24-4176-8540-baf59057aa48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:53.001803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab52356-6a24-4176-8540-baf59057aa48 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:53.145075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1146/2274] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:52 2022] POST /identity/v3/users => generated 293 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:53.146448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:53.157608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ce5e3-7518-4169-9960-53e8d12ba0c3 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:53.162453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e602bf5-a8f8-4d16-99e1-af29d6583e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:53.163153 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e602bf5-a8f8-4d16-99e1-af29d6583e58 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:53.163434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e602bf5-a8f8-4d16-99e1-af29d6583e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:53.164791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e602bf5-a8f8-4d16-99e1-af29d6583e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:53.165499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e602bf5-a8f8-4d16-99e1-af29d6583e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:53.168261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e602bf5-a8f8-4d16-99e1-af29d6583e58 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:53.179573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1129/2275] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:53.184878 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab52356-6a24-4176-8540-baf59057aa48 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:53.211772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab52356-6a24-4176-8540-baf59057aa48 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:53.221226 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5ac23-70b8-46a1-9ed5-140c9f42c80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:53.221842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5ac23-70b8-46a1-9ed5-140c9f42c80f admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:53.222189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5ac23-70b8-46a1-9ed5-140c9f42c80f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:53.222470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5ac23-70b8-46a1-9ed5-140c9f42c80f admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:53.223118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5ac23-70b8-46a1-9ed5-140c9f42c80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:53.224751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5ac23-70b8-46a1-9ed5-140c9f42c80f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:53.239075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1147/2276] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:53.240694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e602bf5-a8f8-4d16-99e1-af29d6583e58 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:53.251866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e602bf5-a8f8-4d16-99e1-af29d6583e58 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:53.256502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91c35e-fbf3-4b1a-b25d-272e68737ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:53.257195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91c35e-fbf3-4b1a-b25d-272e68737ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:53.257477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91c35e-fbf3-4b1a-b25d-272e68737ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:53.257748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91c35e-fbf3-4b1a-b25d-272e68737ba8 admin admin] PATH_INFO: `/v3/projects/3aa94ae6b1fd47c99ee192d4ec3cd065/users/b49e0e1993ba49878e91eb7d9b474624/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:53.274258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91c35e-fbf3-4b1a-b25d-272e68737ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa94ae6b1fd47c99ee192d4ec3cd065, user_id=b49e0e1993ba49878e91eb7d9b474624, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:53.278277 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91c35e-fbf3-4b1a-b25d-272e68737ba8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:53.434824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5ac23-70b8-46a1-9ed5-140c9f42c80f admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:53.436002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1130/2277] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:53 2022] PUT /identity/v3/projects/3aa94ae6b1fd47c99ee192d4ec3cd065/users/b49e0e1993ba49878e91eb7d9b474624/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:53.437569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e3e5ac23-70b8-46a1-9ed5-140c9f42c80f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:53.448683 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004620b6-f3d8-4547-9149-52b48dd42ecc None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:53.449023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004620b6-f3d8-4547-9149-52b48dd42ecc None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:53.449376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004620b6-f3d8-4547-9149-52b48dd42ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:53.458760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5ac23-70b8-46a1-9ed5-140c9f42c80f admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:53.484139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-004620b6-f3d8-4547-9149-52b48dd42ecc None None] MFA Rules not processed for user `b49e0e1993ba49878e91eb7d9b474624`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:53.485616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-004620b6-f3d8-4547-9149-52b48dd42ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:53.545462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:53.546123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:53.546497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:53.546722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] PATH_INFO: `/v3/projects/9961103df0134a7ea78c9074d186e02b` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:53.548614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9961103df0134a7ea78c9074d186e02b)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:53.550616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:53.612338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1131/2278] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:53.613609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004620b6-f3d8-4547-9149-52b48dd42ecc None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:53.627253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004620b6-f3d8-4547-9149-52b48dd42ecc None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:53.632950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240ef5a-4d23-47e6-be05-432cbdbe456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:53.634139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240ef5a-4d23-47e6-be05-432cbdbe456c admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:53.634139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240ef5a-4d23-47e6-be05-432cbdbe456c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:53.634386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240ef5a-4d23-47e6-be05-432cbdbe456c admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:53.635080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240ef5a-4d23-47e6-be05-432cbdbe456c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:53.637309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240ef5a-4d23-47e6-be05-432cbdbe456c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:53.749794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9961103df0134a7ea78c9074d186e02b', 'request_id': 'req-09013795-ce09-4193-8a8c-dc4ac6db6508'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:53.756696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1132/2279] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:53 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:53.760864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240ef5a-4d23-47e6-be05-432cbdbe456c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:53.778540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9961103df0134a7ea78c9074d186e02b', 'request_id': 'req-09013795-ce09-4193-8a8c-dc4ac6db6508'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:53.851760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240ef5a-4d23-47e6-be05-432cbdbe456c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:53.881073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0d71e-3199-46ff-9406-e5cc004f3c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:53.881073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0d71e-3199-46ff-9406-e5cc004f3c61 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:53.881073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0d71e-3199-46ff-9406-e5cc004f3c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:53.881073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0d71e-3199-46ff-9406-e5cc004f3c61 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:53.881931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0d71e-3199-46ff-9406-e5cc004f3c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:53.881931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0d71e-3199-46ff-9406-e5cc004f3c61 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:53.898243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9961103df0134a7ea78c9074d186e02b', 'request_id': 'req-09013795-ce09-4193-8a8c-dc4ac6db6508'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:53.917595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] The token cache is being invalidate because project 9961103df0134a7ea78c9074d186e02b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:53:53.917975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:53.919297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-09013795-ce09-4193-8a8c-dc4ac6db6508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:53:53.921465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1148/2280] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:53:53 2022] DELETE /identity/v3/projects/9961103df0134a7ea78c9074d186e02b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:53.925782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1581f-b3ee-4df2-a1d0-f933e3e5f272 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:53.926140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1581f-b3ee-4df2-a1d0-f933e3e5f272 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:53.926478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1581f-b3ee-4df2-a1d0-f933e3e5f272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:53.950903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a7c1581f-b3ee-4df2-a1d0-f933e3e5f272 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:53.952188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a7c1581f-b3ee-4df2-a1d0-f933e3e5f272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:53.971442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1149/2281] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:53.979522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846a054-4b89-4ecf-85c6-413fe82dcf8e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:53.979815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846a054-4b89-4ecf-85c6-413fe82dcf8e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:53.980085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846a054-4b89-4ecf-85c6-413fe82dcf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:53.996482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1133/2282] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:53 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:54.000368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0d71e-3199-46ff-9406-e5cc004f3c61 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.003919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-20f0d71e-3199-46ff-9406-e5cc004f3c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:54.015340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8846a054-4b89-4ecf-85c6-413fe82dcf8e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:54.017711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8846a054-4b89-4ecf-85c6-413fe82dcf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:54.023852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:54.031995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0d71e-3199-46ff-9406-e5cc004f3c61 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.043478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1150/2283] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:54.044277 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846a054-4b89-4ecf-85c6-413fe82dcf8e None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.045671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe3c3d-e4a1-425d-b11a-db2f003f9c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.046477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe3c3d-e4a1-425d-b11a-db2f003f9c79 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.047054 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe3c3d-e4a1-425d-b11a-db2f003f9c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.047492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe3c3d-e4a1-425d-b11a-db2f003f9c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.048340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe3c3d-e4a1-425d-b11a-db2f003f9c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.050465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe3c3d-e4a1-425d-b11a-db2f003f9c79 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.071096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846a054-4b89-4ecf-85c6-413fe82dcf8e None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.090237 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1134/2284] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:54.091201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba392185-3f60-4cd7-a078-4c4252023c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.091890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba392185-3f60-4cd7-a078-4c4252023c5e admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.092170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba392185-3f60-4cd7-a078-4c4252023c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.092450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba392185-3f60-4cd7-a078-4c4252023c5e admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.098597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe3c3d-e4a1-425d-b11a-db2f003f9c79 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.113161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba392185-3f60-4cd7-a078-4c4252023c5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:53:54.113161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba392185-3f60-4cd7-a078-4c4252023c5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.113161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba392185-3f60-4cd7-a078-4c4252023c5e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.113161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:54.127274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1151/2285] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:53:54.127428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe3c3d-e4a1-425d-b11a-db2f003f9c79 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.136854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8c0bd-1b25-4220-a54d-b642908895ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.136854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8c0bd-1b25-4220-a54d-b642908895ff admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.136854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8c0bd-1b25-4220-a54d-b642908895ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.136854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8c0bd-1b25-4220-a54d-b642908895ff admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.137825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8c0bd-1b25-4220-a54d-b642908895ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.137825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8c0bd-1b25-4220-a54d-b642908895ff admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.139892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba392185-3f60-4cd7-a078-4c4252023c5e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.153988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba392185-3f60-4cd7-a078-4c4252023c5e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.158704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1135/2286] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:53:54.176244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8c0bd-1b25-4220-a54d-b642908895ff admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.181369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3aaa0c-6620-4990-8e7f-210415775bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.182677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aaa0c-6620-4990-8e7f-210415775bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.183191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aaa0c-6620-4990-8e7f-210415775bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.183688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aaa0c-6620-4990-8e7f-210415775bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.184573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3aaa0c-6620-4990-8e7f-210415775bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.187034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8c0bd-1b25-4220-a54d-b642908895ff admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.188710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3aaa0c-6620-4990-8e7f-210415775bf8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.193723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132f09a-9538-4b19-ba2e-4c93a08112a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.194654 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132f09a-9538-4b19-ba2e-4c93a08112a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.195177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132f09a-9538-4b19-ba2e-4c93a08112a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.195645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132f09a-9538-4b19-ba2e-4c93a08112a5 admin admin] PATH_INFO: `/v3/projects/1a0f0950bce94f5196652f3386dbd9fb/users/ef16e9077fd648cca58f3cbc08a0f8dc/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.216215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132f09a-9538-4b19-ba2e-4c93a08112a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0f0950bce94f5196652f3386dbd9fb, user_id=ef16e9077fd648cca58f3cbc08a0f8dc, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.219548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132f09a-9538-4b19-ba2e-4c93a08112a5 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.314981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1136/2287] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:54 2022] PUT /identity/v3/projects/1a0f0950bce94f5196652f3386dbd9fb/users/ef16e9077fd648cca58f3cbc08a0f8dc/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:54.315669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1152/2288] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:54 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:54.319356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132f09a-9538-4b19-ba2e-4c93a08112a5 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.326038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cf3b6-8d74-4bac-b227-9acabcefc45a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.334363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cf3b6-8d74-4bac-b227-9acabcefc45a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.340455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cf3b6-8d74-4bac-b227-9acabcefc45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.379808 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7b7cf3b6-8d74-4bac-b227-9acabcefc45a None None] MFA Rules not processed for user `ef16e9077fd648cca58f3cbc08a0f8dc`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:54.381742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7b7cf3b6-8d74-4bac-b227-9acabcefc45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:54.410834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132f09a-9538-4b19-ba2e-4c93a08112a5 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.467714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1153/2289] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:54.472176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cf3b6-8d74-4bac-b227-9acabcefc45a None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.477517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f4db1-ac89-43dc-b638-d594062e5425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.478106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f4db1-ac89-43dc-b638-d594062e5425 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.479056 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f4db1-ac89-43dc-b638-d594062e5425 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.479332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f4db1-ac89-43dc-b638-d594062e5425 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.480073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f4db1-ac89-43dc-b638-d594062e5425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.482024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f4db1-ac89-43dc-b638-d594062e5425 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.484213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cf3b6-8d74-4bac-b227-9acabcefc45a None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.490124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96488-3d82-475c-a537-e3384d39308f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.490815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96488-3d82-475c-a537-e3384d39308f admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.491175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96488-3d82-475c-a537-e3384d39308f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.491747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f96488-3d82-475c-a537-e3384d39308f admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.492697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f96488-3d82-475c-a537-e3384d39308f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.494700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f96488-3d82-475c-a537-e3384d39308f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.601697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1137/2290] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:54 2022] POST /identity/v3/users => generated 288 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:54.606350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f4db1-ac89-43dc-b638-d594062e5425 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.617706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f4db1-ac89-43dc-b638-d594062e5425 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.623664 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a9b0c-c26d-4236-9fa8-2d2966ceeda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.623949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a9b0c-c26d-4236-9fa8-2d2966ceeda8 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.624243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a9b0c-c26d-4236-9fa8-2d2966ceeda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.624565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a9b0c-c26d-4236-9fa8-2d2966ceeda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.625441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a9b0c-c26d-4236-9fa8-2d2966ceeda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.627280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a9b0c-c26d-4236-9fa8-2d2966ceeda8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.628522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1154/2291] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:53:54 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:53:54.632183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96488-3d82-475c-a537-e3384d39308f admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.641130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1138/2292] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:53:54.643094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a9b0c-c26d-4236-9fa8-2d2966ceeda8 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.654903 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a9b0c-c26d-4236-9fa8-2d2966ceeda8 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.700080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f96488-3d82-475c-a537-e3384d39308f admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.706613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8c00c-f652-4a28-a08e-27744e7c8cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.707238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8c00c-f652-4a28-a08e-27744e7c8cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.707521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8c00c-f652-4a28-a08e-27744e7c8cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.707830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8c00c-f652-4a28-a08e-27744e7c8cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.708475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8c00c-f652-4a28-a08e-27744e7c8cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.710495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8c00c-f652-4a28-a08e-27744e7c8cd7 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.715458 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53134934-38eb-4b10-99c8-51d42506bfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.716070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53134934-38eb-4b10-99c8-51d42506bfe0 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.716347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53134934-38eb-4b10-99c8-51d42506bfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.716623 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53134934-38eb-4b10-99c8-51d42506bfe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.717307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53134934-38eb-4b10-99c8-51d42506bfe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.718950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53134934-38eb-4b10-99c8-51d42506bfe0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.728716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1139/2293] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:53:54.732826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53134934-38eb-4b10-99c8-51d42506bfe0 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.745058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53134934-38eb-4b10-99c8-51d42506bfe0 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.762049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30071dde-afce-489e-8ba4-2b65210f4d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.762049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30071dde-afce-489e-8ba4-2b65210f4d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.762049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30071dde-afce-489e-8ba4-2b65210f4d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.762049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30071dde-afce-489e-8ba4-2b65210f4d01 admin admin] PATH_INFO: `/v3/projects/51069bc6f5b24078a31bf4709a5b2bc4/users/3ba62b853a9440b980d1e2004c71fb65/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.774672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30071dde-afce-489e-8ba4-2b65210f4d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51069bc6f5b24078a31bf4709a5b2bc4, user_id=3ba62b853a9440b980d1e2004c71fb65, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.778831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30071dde-afce-489e-8ba4-2b65210f4d01 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.820729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1155/2294] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:53:54 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:53:54.827909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8c00c-f652-4a28-a08e-27744e7c8cd7 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.837396 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8c00c-f652-4a28-a08e-27744e7c8cd7 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.842678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d47b4-5ceb-424d-8733-02a38b580634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.843257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d47b4-5ceb-424d-8733-02a38b580634 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.843537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d47b4-5ceb-424d-8733-02a38b580634 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.843809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d47b4-5ceb-424d-8733-02a38b580634 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.844435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d47b4-5ceb-424d-8733-02a38b580634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.846031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d47b4-5ceb-424d-8733-02a38b580634 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.856777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1156/2295] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:53:54.860246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d47b4-5ceb-424d-8733-02a38b580634 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:54.870062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d47b4-5ceb-424d-8733-02a38b580634 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:54.874651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31479ad-3edb-45ed-8e32-eabb19a180a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:54.875305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31479ad-3edb-45ed-8e32-eabb19a180a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.875589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31479ad-3edb-45ed-8e32-eabb19a180a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.875909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31479ad-3edb-45ed-8e32-eabb19a180a4 admin admin] PATH_INFO: `/v3/projects/dc88081c408c408b8a98d50167026a55/users/934a7ce6aee44e558207e165c40662ca/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.885163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1140/2296] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:54 2022] PUT /identity/v3/projects/51069bc6f5b24078a31bf4709a5b2bc4/users/3ba62b853a9440b980d1e2004c71fb65/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:54.895835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31479ad-3edb-45ed-8e32-eabb19a180a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc88081c408c408b8a98d50167026a55, user_id=934a7ce6aee44e558207e165c40662ca, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:54.898670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31479ad-3edb-45ed-8e32-eabb19a180a4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:54.903062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319550b5-4318-43c6-9eea-17ebbdc9a22f None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:54.903765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319550b5-4318-43c6-9eea-17ebbdc9a22f None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:54.906428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319550b5-4318-43c6-9eea-17ebbdc9a22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:54.944052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-319550b5-4318-43c6-9eea-17ebbdc9a22f None None] MFA Rules not processed for user `3ba62b853a9440b980d1e2004c71fb65`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:54.948795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-319550b5-4318-43c6-9eea-17ebbdc9a22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:54.982863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1157/2297] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:54 2022] PUT /identity/v3/projects/dc88081c408c408b8a98d50167026a55/users/934a7ce6aee44e558207e165c40662ca/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:53:54.986437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31479ad-3edb-45ed-8e32-eabb19a180a4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:55.038889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1141/2298] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:55.052177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319550b5-4318-43c6-9eea-17ebbdc9a22f None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:55.053630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-319550b5-4318-43c6-9eea-17ebbdc9a22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:55.099797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31479ad-3edb-45ed-8e32-eabb19a180a4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:55.106097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d17b1-2991-402c-a8e4-508eb53f065e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:55.106874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d17b1-2991-402c-a8e4-508eb53f065e admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:55.107323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d17b1-2991-402c-a8e4-508eb53f065e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:55.107945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d17b1-2991-402c-a8e4-508eb53f065e admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:55.108755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d17b1-2991-402c-a8e4-508eb53f065e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:55.110917 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d17b1-2991-402c-a8e4-508eb53f065e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:55.127416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1158/2299] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:53:55.142357 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d17b1-2991-402c-a8e4-508eb53f065e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:55.146664 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319550b5-4318-43c6-9eea-17ebbdc9a22f None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:55.160532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d17b1-2991-402c-a8e4-508eb53f065e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:55.168496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee3439-628e-4850-acc7-c02827fa62a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:55.171308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee3439-628e-4850-acc7-c02827fa62a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:55.171590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee3439-628e-4850-acc7-c02827fa62a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:55.171871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee3439-628e-4850-acc7-c02827fa62a9 admin admin] PATH_INFO: `/v3/projects/dc88081c408c408b8a98d50167026a55/users/934a7ce6aee44e558207e165c40662ca/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:55.176236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee3439-628e-4850-acc7-c02827fa62a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc88081c408c408b8a98d50167026a55, user_id=934a7ce6aee44e558207e165c40662ca, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:55.179454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee3439-628e-4850-acc7-c02827fa62a9 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:55.194371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1159/2300] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:53:55 2022] PUT /identity/v3/projects/dc88081c408c408b8a98d50167026a55/users/934a7ce6aee44e558207e165c40662ca/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:53:55.206172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d142a47-6415-402e-b536-d129e6f4a11e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:55.206454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d142a47-6415-402e-b536-d129e6f4a11e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:55.206738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d142a47-6415-402e-b536-d129e6f4a11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:55.233784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e6983-61f2-4651-825d-4c319c430897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:55.234851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e6983-61f2-4651-825d-4c319c430897 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:55.235210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e6983-61f2-4651-825d-4c319c430897 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:55.235551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e6983-61f2-4651-825d-4c319c430897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:55.240422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4d142a47-6415-402e-b536-d129e6f4a11e None None] MFA Rules not processed for user `934a7ce6aee44e558207e165c40662ca`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:55.241877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4d142a47-6415-402e-b536-d129e6f4a11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:55.304416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e6983-61f2-4651-825d-4c319c430897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:55.305908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e6983-61f2-4651-825d-4c319c430897 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:55.316225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1142/2301] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:53:55.320918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1160/2302] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:53:55.333273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e6983-61f2-4651-825d-4c319c430897 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:55.337378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ee1e6983-61f2-4651-825d-4c319c430897 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:55.345818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e6983-61f2-4651-825d-4c319c430897 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:55.398392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb51a6-d5d1-488a-a59c-6a7accaa76f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:55.399010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb51a6-d5d1-488a-a59c-6a7accaa76f4 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:55.399291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb51a6-d5d1-488a-a59c-6a7accaa76f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:55.399564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb51a6-d5d1-488a-a59c-6a7accaa76f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:55.401992 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bdbb51a6-d5d1-488a-a59c-6a7accaa76f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:55.427029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:55.487999 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb51a6-d5d1-488a-a59c-6a7accaa76f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:55.489851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb51a6-d5d1-488a-a59c-6a7accaa76f4 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:55.502710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1143/2303] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:55.632235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:56.598966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fac0c0-ecaf-4041-a208-99c3b17bc718 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:56.599502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fac0c0-ecaf-4041-a208-99c3b17bc718 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:56.599793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fac0c0-ecaf-4041-a208-99c3b17bc718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:56.625012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-40fac0c0-ecaf-4041-a208-99c3b17bc718 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:56.626245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-40fac0c0-ecaf-4041-a208-99c3b17bc718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:56.663152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:56.713651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1161/2304] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:53:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:56.774350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb51a6-d5d1-488a-a59c-6a7accaa76f4 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:56.776619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bdbb51a6-d5d1-488a-a59c-6a7accaa76f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:56.862420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb51a6-d5d1-488a-a59c-6a7accaa76f4 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:56.867570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf54d84-e3b2-4988-88f2-a9f7f6b481bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:56.868184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf54d84-e3b2-4988-88f2-a9f7f6b481bf service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:56.868478 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf54d84-e3b2-4988-88f2-a9f7f6b481bf service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:56.868755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf54d84-e3b2-4988-88f2-a9f7f6b481bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:56.878323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf54d84-e3b2-4988-88f2-a9f7f6b481bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:56.886947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf54d84-e3b2-4988-88f2-a9f7f6b481bf service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:56.888248 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1144/2305] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:53:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 18:53:57.122902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890e251-7250-4916-8be9-073d4d49d975 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:57.123191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890e251-7250-4916-8be9-073d4d49d975 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:57.123470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890e251-7250-4916-8be9-073d4d49d975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:57.137738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:57.144049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-0890e251-7250-4916-8be9-073d4d49d975 None None] MFA Rules not processed for user `91e852d04f9f464ab8afbaeb9ff355e6`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:53:57.145395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0890e251-7250-4916-8be9-073d4d49d975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:57.207127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1162/2306] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:53:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:53:58.209369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:53:58.409341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf54d84-e3b2-4988-88f2-a9f7f6b481bf service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:58.428018 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf54d84-e3b2-4988-88f2-a9f7f6b481bf service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:58.438442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37c4fc-2c55-4b15-bcf6-9b4fe3a2ea63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:58.439381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37c4fc-2c55-4b15-bcf6-9b4fe3a2ea63 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:58.439939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37c4fc-2c55-4b15-bcf6-9b4fe3a2ea63 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:58.440456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37c4fc-2c55-4b15-bcf6-9b4fe3a2ea63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:58.625487 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37c4fc-2c55-4b15-bcf6-9b4fe3a2ea63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:58.627199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37c4fc-2c55-4b15-bcf6-9b4fe3a2ea63 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:58.665205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1145/2307] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:53:58.778256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890e251-7250-4916-8be9-073d4d49d975 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:53:58.778256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0890e251-7250-4916-8be9-073d4d49d975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:58.898002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890e251-7250-4916-8be9-073d4d49d975 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:53:58.909756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca383d-dd46-415c-a6f8-9917d647d414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:53:58.910788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca383d-dd46-415c-a6f8-9917d647d414 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:53:58.910916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca383d-dd46-415c-a6f8-9917d647d414 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:53:58.911155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca383d-dd46-415c-a6f8-9917d647d414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:53:58.912995 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ceca383d-dd46-415c-a6f8-9917d647d414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:53:59.065621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca383d-dd46-415c-a6f8-9917d647d414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:53:59.067458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca383d-dd46-415c-a6f8-9917d647d414 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:53:59.081215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1163/2308] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:53:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:02.060168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37c4fc-2c55-4b15-bcf6-9b4fe3a2ea63 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:02.074448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37c4fc-2c55-4b15-bcf6-9b4fe3a2ea63 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:02.089016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384d77a-13b3-49f2-b928-5b9c5efe3d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:02.089744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384d77a-13b3-49f2-b928-5b9c5efe3d50 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:02.090029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384d77a-13b3-49f2-b928-5b9c5efe3d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:02.090304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384d77a-13b3-49f2-b928-5b9c5efe3d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:02.126571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384d77a-13b3-49f2-b928-5b9c5efe3d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:02.128448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384d77a-13b3-49f2-b928-5b9c5efe3d50 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:02.156860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1146/2309] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:02.933019 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca383d-dd46-415c-a6f8-9917d647d414 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:03.029650 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca383d-dd46-415c-a6f8-9917d647d414 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:03.036969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:03.037754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:03.038110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:03.038458 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] PATH_INFO: `/v3/users/3ba62b853a9440b980d1e2004c71fb65` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:03.039837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba62b853a9440b980d1e2004c71fb65)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:03.043273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:03.046153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:03.254179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba62b853a9440b980d1e2004c71fb65'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:03.255484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba62b853a9440b980d1e2004c71fb65'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:03.258788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba62b853a9440b980d1e2004c71fb65'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:03.280938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba62b853a9440b980d1e2004c71fb65'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:03.317910 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba62b853a9440b980d1e2004c71fb65'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:03.435198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1164/2310] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:54:02 2022] DELETE /identity/v3/users/3ba62b853a9440b980d1e2004c71fb65 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:03.439797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f1bf5-6072-40f6-aa97-f5d321127370 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:03.440908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f1bf5-6072-40f6-aa97-f5d321127370 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:03.441310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f1bf5-6072-40f6-aa97-f5d321127370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:03.643864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-bb3f1bf5-6072-40f6-aa97-f5d321127370 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:03.645135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bb3f1bf5-6072-40f6-aa97-f5d321127370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:03.763666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1147/2311] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:03.773588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:03.775540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:03.874689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4332621-0249-40ee-af2a-ffb64e41339d admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:03.878789 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93aa7d3-9980-4529-9da2-2e0861e478f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:03.879415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93aa7d3-9980-4529-9da2-2e0861e478f5 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:03.879819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93aa7d3-9980-4529-9da2-2e0861e478f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:03.883242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93aa7d3-9980-4529-9da2-2e0861e478f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:03.900482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93aa7d3-9980-4529-9da2-2e0861e478f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:03.903041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93aa7d3-9980-4529-9da2-2e0861e478f5 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:03.925169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1165/2312] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:54:04.728681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f1bf5-6072-40f6-aa97-f5d321127370 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:04.731771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bb3f1bf5-6072-40f6-aa97-f5d321127370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:04.752427 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f1bf5-6072-40f6-aa97-f5d321127370 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:04.769795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:04.770415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:04.770703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:04.771002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] PATH_INFO: `/v3/projects/51069bc6f5b24078a31bf4709a5b2bc4` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:04.772244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51069bc6f5b24078a31bf4709a5b2bc4)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:04.777328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:04.901824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51069bc6f5b24078a31bf4709a5b2bc4', 'request_id': 'req-3cc68e82-e5a2-4498-ab59-a64ddee505c6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:04.917193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51069bc6f5b24078a31bf4709a5b2bc4', 'request_id': 'req-3cc68e82-e5a2-4498-ab59-a64ddee505c6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:04.926655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51069bc6f5b24078a31bf4709a5b2bc4', 'request_id': 'req-3cc68e82-e5a2-4498-ab59-a64ddee505c6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:05.191683 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] The token cache is being invalidate because project 51069bc6f5b24078a31bf4709a5b2bc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:54:05.191987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:05.192640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3cc68e82-e5a2-4498-ab59-a64ddee505c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:05.194265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1148/2313] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:54:04 2022] DELETE /identity/v3/projects/51069bc6f5b24078a31bf4709a5b2bc4 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:10.114640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93aa7d3-9980-4529-9da2-2e0861e478f5 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:10.116320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e93aa7d3-9980-4529-9da2-2e0861e478f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:10.137620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93aa7d3-9980-4529-9da2-2e0861e478f5 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:10.245875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:10.246904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:10.247280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:10.247633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] PATH_INFO: `/v3/users/24f1de8878174b36ae83b2afdc960b84` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:10.251406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f1de8878174b36ae83b2afdc960b84)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:10.253504 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:10.399155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:10.530875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f1de8878174b36ae83b2afdc960b84'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:10.531587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f1de8878174b36ae83b2afdc960b84'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:10.532211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f1de8878174b36ae83b2afdc960b84'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:10.549844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f1de8878174b36ae83b2afdc960b84'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:10.577920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f1de8878174b36ae83b2afdc960b84'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:10.735264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1166/2314] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:54:10 2022] DELETE /identity/v3/users/24f1de8878174b36ae83b2afdc960b84 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:10.740145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d537-009f-46b5-a4f7-5db5c1c48f33 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:10.741174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d537-009f-46b5-a4f7-5db5c1c48f33 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:10.742087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d537-009f-46b5-a4f7-5db5c1c48f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:10.765978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-be76d537-009f-46b5-a4f7-5db5c1c48f33 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:10.767573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-be76d537-009f-46b5-a4f7-5db5c1c48f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:10.772154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:10.857664 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1149/2315] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:10.872828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:10.878942 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:10.958199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853712d-1d8a-4e42-9c74-f8da5ea42816 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:10.963141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a1f6e-9bf1-4e89-b258-d31b577cf2df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:10.963849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a1f6e-9bf1-4e89-b258-d31b577cf2df service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:10.964139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a1f6e-9bf1-4e89-b258-d31b577cf2df service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:10.964421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a1f6e-9bf1-4e89-b258-d31b577cf2df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:10.975001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a1f6e-9bf1-4e89-b258-d31b577cf2df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:10.976434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a1f6e-9bf1-4e89-b258-d31b577cf2df service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:10.987333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1167/2316] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:11.496278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d537-009f-46b5-a4f7-5db5c1c48f33 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:11.501021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-be76d537-009f-46b5-a4f7-5db5c1c48f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:11.512833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d537-009f-46b5-a4f7-5db5c1c48f33 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:11.518008 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:11.518649 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:11.518928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:11.519204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] PATH_INFO: `/v3/projects/0aadff73d43243d794a8552eac667e4f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:11.520324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aadff73d43243d794a8552eac667e4f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:11.526378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:11.638775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aadff73d43243d794a8552eac667e4f', 'request_id': 'req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:11.645621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aadff73d43243d794a8552eac667e4f', 'request_id': 'req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:11.652458 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aadff73d43243d794a8552eac667e4f', 'request_id': 'req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:11.734603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] The token cache is being invalidate because project 0aadff73d43243d794a8552eac667e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:54:11.741969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:11.741969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:11.741969 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1150/2317] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:54:11 2022] DELETE /identity/v3/projects/0aadff73d43243d794a8552eac667e4f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:11.743098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091c404-cae9-46cc-b6d6-75854d9b1f88 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:11.743098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091c404-cae9-46cc-b6d6-75854d9b1f88 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:11.743098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091c404-cae9-46cc-b6d6-75854d9b1f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:11.763439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4091c404-cae9-46cc-b6d6-75854d9b1f88 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:11.770761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4091c404-cae9-46cc-b6d6-75854d9b1f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:11.844201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1168/2318] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:11.846735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:11.848379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:11.852401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28488c-38f7-4a90-a6fa-08b866c36bd9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:11.852844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28488c-38f7-4a90-a6fa-08b866c36bd9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:11.853162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28488c-38f7-4a90-a6fa-08b866c36bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:11.866550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f3553-4a78-4c2d-9f20-89ce8b74ac5a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:11.873571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7d28488c-38f7-4a90-a6fa-08b866c36bd9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:11.878336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:11.878492 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7d28488c-38f7-4a90-a6fa-08b866c36bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:11.878601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:11.878601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:11.878601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] PATH_INFO: `/v3/users/b49e0e1993ba49878e91eb7d9b474624` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:11.878601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49e0e1993ba49878e91eb7d9b474624)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:11.879476 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:11.885223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1169/2319] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:11.890363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28488c-38f7-4a90-a6fa-08b866c36bd9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:11.907030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28488c-38f7-4a90-a6fa-08b866c36bd9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:11.912313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246000b7-fff1-428b-b397-4d2990ee2db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:11.912921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246000b7-fff1-428b-b397-4d2990ee2db6 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:11.913231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246000b7-fff1-428b-b397-4d2990ee2db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:11.913506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246000b7-fff1-428b-b397-4d2990ee2db6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:11.914150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246000b7-fff1-428b-b397-4d2990ee2db6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:54:11.914450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246000b7-fff1-428b-b397-4d2990ee2db6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:11.916692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246000b7-fff1-428b-b397-4d2990ee2db6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:11.928952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1170/2320] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:54:11.934030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246000b7-fff1-428b-b397-4d2990ee2db6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:11.944194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246000b7-fff1-428b-b397-4d2990ee2db6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:11.950013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52449b17-67a4-4901-9205-23b68689ab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:11.950777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52449b17-67a4-4901-9205-23b68689ab58 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:11.951161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52449b17-67a4-4901-9205-23b68689ab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:11.953166 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52449b17-67a4-4901-9205-23b68689ab58 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:11.953166 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52449b17-67a4-4901-9205-23b68689ab58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:11.954033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52449b17-67a4-4901-9205-23b68689ab58 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:12.006512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49e0e1993ba49878e91eb7d9b474624'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:12.007304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49e0e1993ba49878e91eb7d9b474624'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:12.008087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49e0e1993ba49878e91eb7d9b474624'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:12.017793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1171/2321] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:54:11 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:54:12.021884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52449b17-67a4-4901-9205-23b68689ab58 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:12.024061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-52449b17-67a4-4901-9205-23b68689ab58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:12.024444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49e0e1993ba49878e91eb7d9b474624'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:12.030989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:12.034583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e9027e2c-0155-4118-b17f-ebba2200540a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49e0e1993ba49878e91eb7d9b474624'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:12.092457 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52449b17-67a4-4901-9205-23b68689ab58 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:12.097358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c4b0d-13c4-47eb-b000-aea2577a545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:12.097978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c4b0d-13c4-47eb-b000-aea2577a545a admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:12.098444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c4b0d-13c4-47eb-b000-aea2577a545a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:12.098565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c4b0d-13c4-47eb-b000-aea2577a545a admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:12.099196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c4b0d-13c4-47eb-b000-aea2577a545a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:12.101307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c4b0d-13c4-47eb-b000-aea2577a545a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:12.109877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1151/2322] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:54:11 2022] DELETE /identity/v3/users/b49e0e1993ba49878e91eb7d9b474624 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:12.114220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec783d-9394-4c29-984f-cf09f3419302 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:12.114582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec783d-9394-4c29-984f-cf09f3419302 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:12.114955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec783d-9394-4c29-984f-cf09f3419302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:12.124054 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:12.137197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ecec783d-9394-4c29-984f-cf09f3419302 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:12.138453 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ecec783d-9394-4c29-984f-cf09f3419302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:12.197693 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1152/2323] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:54:12.205078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1172/2324] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:54:12 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:54:12.210334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec783d-9394-4c29-984f-cf09f3419302 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:12.210334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ecec783d-9394-4c29-984f-cf09f3419302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:12.214376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c4b0d-13c4-47eb-b000-aea2577a545a admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:12.224589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c4b0d-13c4-47eb-b000-aea2577a545a admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:12.229568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb9a6e-ed69-4354-bc10-5270cb120db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:12.236548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb9a6e-ed69-4354-bc10-5270cb120db5 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:12.236548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb9a6e-ed69-4354-bc10-5270cb120db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:12.236548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb9a6e-ed69-4354-bc10-5270cb120db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:12.236548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb9a6e-ed69-4354-bc10-5270cb120db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:12.236548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb9a6e-ed69-4354-bc10-5270cb120db5 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:12.243716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1173/2325] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:54:12.248024 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb9a6e-ed69-4354-bc10-5270cb120db5 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:12.258249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb9a6e-ed69-4354-bc10-5270cb120db5 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:12.263794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba29e82-5935-411f-bf79-7ed3ca2d4eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:12.264388 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba29e82-5935-411f-bf79-7ed3ca2d4eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:12.264672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba29e82-5935-411f-bf79-7ed3ca2d4eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:12.264950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba29e82-5935-411f-bf79-7ed3ca2d4eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:12.265623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba29e82-5935-411f-bf79-7ed3ca2d4eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:12.267222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba29e82-5935-411f-bf79-7ed3ca2d4eb2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:12.270361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec783d-9394-4c29-984f-cf09f3419302 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:12.278475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c78415-da09-4380-b926-bc927415cd84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:12.278475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c78415-da09-4380-b926-bc927415cd84 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:12.278475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c78415-da09-4380-b926-bc927415cd84 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:12.278475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c78415-da09-4380-b926-bc927415cd84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:12.279701 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1174/2326] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:54:12.283867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba29e82-5935-411f-bf79-7ed3ca2d4eb2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:12.286745 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c78415-da09-4380-b926-bc927415cd84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:12.290378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c78415-da09-4380-b926-bc927415cd84 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:12.293872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba29e82-5935-411f-bf79-7ed3ca2d4eb2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:12.299093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b211ee-dfa6-43ce-b344-f1925d9e77b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:12.299754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b211ee-dfa6-43ce-b344-f1925d9e77b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:12.300044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b211ee-dfa6-43ce-b344-f1925d9e77b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:12.301628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b211ee-dfa6-43ce-b344-f1925d9e77b3 admin admin] PATH_INFO: `/v3/projects/a9c50a271b504ced86c125625e44c19e/users/f7555f03f8e34998ab1890279bf9b5c8/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:12.313160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1153/2327] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:54:12.320071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b211ee-dfa6-43ce-b344-f1925d9e77b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c50a271b504ced86c125625e44c19e, user_id=f7555f03f8e34998ab1890279bf9b5c8, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:12.323405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b211ee-dfa6-43ce-b344-f1925d9e77b3 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:12.386545 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1175/2328] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:54:12 2022] PUT /identity/v3/projects/a9c50a271b504ced86c125625e44c19e/users/f7555f03f8e34998ab1890279bf9b5c8/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:54:12.391531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02c68e-4913-4d2e-afcc-39e5629acca8 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:12.391816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02c68e-4913-4d2e-afcc-39e5629acca8 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:12.392373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02c68e-4913-4d2e-afcc-39e5629acca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:12.413107 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-3f02c68e-4913-4d2e-afcc-39e5629acca8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:12.414331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3f02c68e-4913-4d2e-afcc-39e5629acca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:12.475057 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1154/2329] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:12.486132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b211ee-dfa6-43ce-b344-f1925d9e77b3 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:12.488924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-84b211ee-dfa6-43ce-b344-f1925d9e77b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:12.555326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b211ee-dfa6-43ce-b344-f1925d9e77b3 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:12.559558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abb69e-9057-4a5f-9bbc-f4d18b2cc981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:12.560169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abb69e-9057-4a5f-9bbc-f4d18b2cc981 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:12.560447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abb69e-9057-4a5f-9bbc-f4d18b2cc981 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:12.560719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abb69e-9057-4a5f-9bbc-f4d18b2cc981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:12.569837 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abb69e-9057-4a5f-9bbc-f4d18b2cc981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:12.571270 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abb69e-9057-4a5f-9bbc-f4d18b2cc981 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:12.581809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1176/2330] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:54:12.724137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02c68e-4913-4d2e-afcc-39e5629acca8 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:12.725698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3f02c68e-4913-4d2e-afcc-39e5629acca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:12.738276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02c68e-4913-4d2e-afcc-39e5629acca8 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:12.748911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:12.749627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:12.749953 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:12.750187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] PATH_INFO: `/v3/projects/3aa94ae6b1fd47c99ee192d4ec3cd065` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:12.751320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aa94ae6b1fd47c99ee192d4ec3cd065)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:12.753291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:12.855522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:12.859209 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aa94ae6b1fd47c99ee192d4ec3cd065', 'request_id': 'req-35a4a7fd-c336-45c7-9899-1472229fbb4a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:12.865942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aa94ae6b1fd47c99ee192d4ec3cd065', 'request_id': 'req-35a4a7fd-c336-45c7-9899-1472229fbb4a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:12.871902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aa94ae6b1fd47c99ee192d4ec3cd065', 'request_id': 'req-35a4a7fd-c336-45c7-9899-1472229fbb4a'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:12.973747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] The token cache is being invalidate because project 3aa94ae6b1fd47c99ee192d4ec3cd065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:54:12.974063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:12.974684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:12.976299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1155/2331] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:54:12 2022] DELETE /identity/v3/projects/3aa94ae6b1fd47c99ee192d4ec3cd065 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:12.980581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abb69e-9057-4a5f-9bbc-f4d18b2cc981 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:12.990785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abb69e-9057-4a5f-9bbc-f4d18b2cc981 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:13.040284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:13.050363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:13.052307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:13.052307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:13.052307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] PATH_INFO: `/v3/users/ef16e9077fd648cca58f3cbc08a0f8dc` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:13.052739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef16e9077fd648cca58f3cbc08a0f8dc)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:13.054706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:13.221924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef16e9077fd648cca58f3cbc08a0f8dc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:13.222619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef16e9077fd648cca58f3cbc08a0f8dc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:13.223340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef16e9077fd648cca58f3cbc08a0f8dc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:13.237738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef16e9077fd648cca58f3cbc08a0f8dc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:13.252149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef16e9077fd648cca58f3cbc08a0f8dc'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:13.334929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1177/2332] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:54:12 2022] DELETE /identity/v3/users/ef16e9077fd648cca58f3cbc08a0f8dc => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:13.804876 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:13.808169 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:13.824835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a7fd-c336-45c7-9899-1472229fbb4a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:13.892011 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:13.892786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:13.893238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:13.893512 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] PATH_INFO: `/v3/projects/1a0f0950bce94f5196652f3386dbd9fb` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:13.894710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0f0950bce94f5196652f3386dbd9fb)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:13.896765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:14.016408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0f0950bce94f5196652f3386dbd9fb', 'request_id': 'req-287756b8-a8b0-4eca-8ce3-f9e6555294ed'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:14.024191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0f0950bce94f5196652f3386dbd9fb', 'request_id': 'req-287756b8-a8b0-4eca-8ce3-f9e6555294ed'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:14.030102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0f0950bce94f5196652f3386dbd9fb', 'request_id': 'req-287756b8-a8b0-4eca-8ce3-f9e6555294ed'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:14.100857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] The token cache is being invalidate because project 1a0f0950bce94f5196652f3386dbd9fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:54:14.101336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:14.103909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:14.109326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1156/2333] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:54:13 2022] DELETE /identity/v3/projects/1a0f0950bce94f5196652f3386dbd9fb => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:14.111539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:14.113552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:14.136289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1afa05-c881-4eb8-8415-64c8a6f4220c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:14.237141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:14.238258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:14.238770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:14.240207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] PATH_INFO: `/v3/users/934a7ce6aee44e558207e165c40662ca` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:14.245067 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=934a7ce6aee44e558207e165c40662ca)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:14.248310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:14.512401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934a7ce6aee44e558207e165c40662ca'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:14.513400 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934a7ce6aee44e558207e165c40662ca'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:14.514244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934a7ce6aee44e558207e165c40662ca'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:14.522495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934a7ce6aee44e558207e165c40662ca'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:14.541034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:14.543811 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3f1fffca-f8a3-4924-a131-d3f087c7b890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934a7ce6aee44e558207e165c40662ca'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:14.676048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1178/2334] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:54:14 2022] DELETE /identity/v3/users/934a7ce6aee44e558207e165c40662ca => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:15.424818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:15.426348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:15.476449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287756b8-a8b0-4eca-8ce3-f9e6555294ed admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:15.568884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:15.569740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:15.570021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:15.570319 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] PATH_INFO: `/v3/projects/dc88081c408c408b8a98d50167026a55` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:15.571436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc88081c408c408b8a98d50167026a55)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:15.573515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:15.576138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:15.718219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc88081c408c408b8a98d50167026a55', 'request_id': 'req-438b6050-d9d4-479d-9373-cc1015098e29'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:15.724960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc88081c408c408b8a98d50167026a55', 'request_id': 'req-438b6050-d9d4-479d-9373-cc1015098e29'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:15.734058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc88081c408c408b8a98d50167026a55', 'request_id': 'req-438b6050-d9d4-479d-9373-cc1015098e29'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:15.799032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] The token cache is being invalidate because project dc88081c408c408b8a98d50167026a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:54:15.799437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:15.804863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-438b6050-d9d4-479d-9373-cc1015098e29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:15.807275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1157/2335] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:54:15 2022] DELETE /identity/v3/projects/dc88081c408c408b8a98d50167026a55 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:15.812769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85451839-e1f0-4833-bcab-5036d79ec988 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:15.814288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85451839-e1f0-4833-bcab-5036d79ec988 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:15.814573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85451839-e1f0-4833-bcab-5036d79ec988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:15.836255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-85451839-e1f0-4833-bcab-5036d79ec988 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:15.838353 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-85451839-e1f0-4833-bcab-5036d79ec988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:15.907925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1179/2336] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:15.916406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def9b30-ea13-4776-94de-1e732e532da7 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:15.916690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def9b30-ea13-4776-94de-1e732e532da7 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:15.916995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def9b30-ea13-4776-94de-1e732e532da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:15.943590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-3def9b30-ea13-4776-94de-1e732e532da7 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:15.944170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3def9b30-ea13-4776-94de-1e732e532da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:15.974097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1158/2337] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:15.978210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85451839-e1f0-4833-bcab-5036d79ec988 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:15.989666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85451839-e1f0-4833-bcab-5036d79ec988 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:15.995453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabbdf3-17c5-485f-bc08-4c86dd53fec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:15.996095 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabbdf3-17c5-485f-bc08-4c86dd53fec4 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:15.996378 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabbdf3-17c5-485f-bc08-4c86dd53fec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:15.996657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabbdf3-17c5-485f-bc08-4c86dd53fec4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:15.997534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabbdf3-17c5-485f-bc08-4c86dd53fec4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:54:15.997852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabbdf3-17c5-485f-bc08-4c86dd53fec4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:15.999487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabbdf3-17c5-485f-bc08-4c86dd53fec4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:16.026287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1180/2338] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:54:16.031500 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def9b30-ea13-4776-94de-1e732e532da7 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:16.046257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def9b30-ea13-4776-94de-1e732e532da7 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:16.054371 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5c6cd-988c-4b57-ab16-8a416add5381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:16.055303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5c6cd-988c-4b57-ab16-8a416add5381 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:16.055880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5c6cd-988c-4b57-ab16-8a416add5381 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:16.057927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5c6cd-988c-4b57-ab16-8a416add5381 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:16.058864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5c6cd-988c-4b57-ab16-8a416add5381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:16.061003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5c6cd-988c-4b57-ab16-8a416add5381 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:16.179752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1159/2339] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:54:16 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:54:16.184575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabbdf3-17c5-485f-bc08-4c86dd53fec4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:16.207251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabbdf3-17c5-485f-bc08-4c86dd53fec4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:16.275650 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c326-e113-47d9-9dea-a75d00a47397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:16.276472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c326-e113-47d9-9dea-a75d00a47397 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:16.276766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c326-e113-47d9-9dea-a75d00a47397 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:16.277068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c326-e113-47d9-9dea-a75d00a47397 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:16.277905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453c326-e113-47d9-9dea-a75d00a47397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:16.280075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453c326-e113-47d9-9dea-a75d00a47397 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:16.405827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1181/2340] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:54:16 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:54:16.407208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5c6cd-988c-4b57-ab16-8a416add5381 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:16.419222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5c6cd-988c-4b57-ab16-8a416add5381 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:16.424351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4f81c-307a-4994-9be2-463ba278fdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:16.425144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4f81c-307a-4994-9be2-463ba278fdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:16.425766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4f81c-307a-4994-9be2-463ba278fdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:16.425766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4f81c-307a-4994-9be2-463ba278fdb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:16.426348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4f81c-307a-4994-9be2-463ba278fdb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:16.428113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4f81c-307a-4994-9be2-463ba278fdb1 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:16.439603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1160/2341] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:54:16.449415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c326-e113-47d9-9dea-a75d00a47397 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:16.475820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c326-e113-47d9-9dea-a75d00a47397 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:16.483622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1887b-7f35-4e50-9d6d-0ee4bb6c6255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:16.484257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1887b-7f35-4e50-9d6d-0ee4bb6c6255 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:16.484540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1887b-7f35-4e50-9d6d-0ee4bb6c6255 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:16.484907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1887b-7f35-4e50-9d6d-0ee4bb6c6255 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:16.486363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1887b-7f35-4e50-9d6d-0ee4bb6c6255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:16.487462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1887b-7f35-4e50-9d6d-0ee4bb6c6255 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:16.500958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1182/2342] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:54:16.505428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4f81c-307a-4994-9be2-463ba278fdb1 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:16.517563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4f81c-307a-4994-9be2-463ba278fdb1 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:16.523125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ec89b-9323-4660-b4d0-9ff3e10b367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:16.523797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ec89b-9323-4660-b4d0-9ff3e10b367d admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:16.524073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ec89b-9323-4660-b4d0-9ff3e10b367d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:16.524504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ec89b-9323-4660-b4d0-9ff3e10b367d admin admin] PATH_INFO: `/v3/projects/46c3deb0f3f64116ba78bb2e71b3bc7c/users/ecd17ab0627e4afbb7b165d974b27759/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:16.545530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ec89b-9323-4660-b4d0-9ff3e10b367d admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c3deb0f3f64116ba78bb2e71b3bc7c, user_id=ecd17ab0627e4afbb7b165d974b27759, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:16.548206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ec89b-9323-4660-b4d0-9ff3e10b367d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:16.630354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1161/2343] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:54:16 2022] PUT /identity/v3/projects/46c3deb0f3f64116ba78bb2e71b3bc7c/users/ecd17ab0627e4afbb7b165d974b27759/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:54:16.640274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39199b-4288-4662-932e-4f53826c49b0 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:16.640561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39199b-4288-4662-932e-4f53826c49b0 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:16.640836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39199b-4288-4662-932e-4f53826c49b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:16.691308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-cd39199b-4288-4662-932e-4f53826c49b0 None None] MFA Rules not processed for user `ecd17ab0627e4afbb7b165d974b27759`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:16.692842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-cd39199b-4288-4662-932e-4f53826c49b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:16.759805 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:16.787184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1183/2344] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:16.799743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ec89b-9323-4660-b4d0-9ff3e10b367d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:16.802407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-799ec89b-9323-4660-b4d0-9ff3e10b367d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:16.898420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ec89b-9323-4660-b4d0-9ff3e10b367d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:16.904912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9b176-8b0a-4887-88a3-e5e9480e17ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:16.905939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9b176-8b0a-4887-88a3-e5e9480e17ce service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:16.906461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9b176-8b0a-4887-88a3-e5e9480e17ce service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:16.906837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9b176-8b0a-4887-88a3-e5e9480e17ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:16.919123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9b176-8b0a-4887-88a3-e5e9480e17ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:16.920666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9b176-8b0a-4887-88a3-e5e9480e17ce service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:16.935115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1162/2345] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:18.944573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1c474-c34a-4225-bfbd-29d52e13c605 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:18.945120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1c474-c34a-4225-bfbd-29d52e13c605 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:18.945782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1c474-c34a-4225-bfbd-29d52e13c605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:19.013252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-84e1c474-c34a-4225-bfbd-29d52e13c605 None None] MFA Rules not processed for user `f7555f03f8e34998ab1890279bf9b5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:19.014704 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-84e1c474-c34a-4225-bfbd-29d52e13c605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:19.248015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1184/2346] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:54:19.258301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9b176-8b0a-4887-88a3-e5e9480e17ce service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:19.260529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d5a9b176-8b0a-4887-88a3-e5e9480e17ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:19.303149 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:19.338321 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9b176-8b0a-4887-88a3-e5e9480e17ce service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:19.339064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976267b-c0eb-4b18-8cf8-3fa13cc7b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:19.339785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976267b-c0eb-4b18-8cf8-3fa13cc7b3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:19.340070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976267b-c0eb-4b18-8cf8-3fa13cc7b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:19.340345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976267b-c0eb-4b18-8cf8-3fa13cc7b3b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:19.341016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976267b-c0eb-4b18-8cf8-3fa13cc7b3b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:19.343123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976267b-c0eb-4b18-8cf8-3fa13cc7b3b7 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:19.505332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1163/2347] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:54:19 2022] POST /identity/v3/projects => generated 383 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:54:19.510059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1c474-c34a-4225-bfbd-29d52e13c605 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:19.526102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1c474-c34a-4225-bfbd-29d52e13c605 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:19.612834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323d26d-970b-4d04-b860-b9daffc7c410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:19.613513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d26d-970b-4d04-b860-b9daffc7c410 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:19.613800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d26d-970b-4d04-b860-b9daffc7c410 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:19.614076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d26d-970b-4d04-b860-b9daffc7c410 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:19.614709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323d26d-970b-4d04-b860-b9daffc7c410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:19.616647 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323d26d-970b-4d04-b860-b9daffc7c410 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:19.711639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1185/2348] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:54:19 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:54:19.716407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976267b-c0eb-4b18-8cf8-3fa13cc7b3b7 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:19.726905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976267b-c0eb-4b18-8cf8-3fa13cc7b3b7 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:19.732705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a50779-461f-450e-a7d0-d8da1a860f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:19.733362 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a50779-461f-450e-a7d0-d8da1a860f91 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:19.733649 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a50779-461f-450e-a7d0-d8da1a860f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:19.733934 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a50779-461f-450e-a7d0-d8da1a860f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:19.734586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a50779-461f-450e-a7d0-d8da1a860f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:19.736344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a50779-461f-450e-a7d0-d8da1a860f91 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:19.750993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1164/2349] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:54:19.755685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323d26d-970b-4d04-b860-b9daffc7c410 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:19.767034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323d26d-970b-4d04-b860-b9daffc7c410 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:19.771660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee89532-fa69-42e6-901e-7d4968e163fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:19.772363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee89532-fa69-42e6-901e-7d4968e163fc admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:19.772623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee89532-fa69-42e6-901e-7d4968e163fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:19.772893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee89532-fa69-42e6-901e-7d4968e163fc admin admin] PATH_INFO: `/v3/projects/aac7e6ed26fa439bb2d44d462b70c6d9/users/72f7e33f9a06403ab2c3c5fb553f6978/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:19.805454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee89532-fa69-42e6-901e-7d4968e163fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac7e6ed26fa439bb2d44d462b70c6d9, user_id=72f7e33f9a06403ab2c3c5fb553f6978, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:19.810319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee89532-fa69-42e6-901e-7d4968e163fc admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:19.867632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1186/2350] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:54:19 2022] PUT /identity/v3/projects/aac7e6ed26fa439bb2d44d462b70c6d9/users/72f7e33f9a06403ab2c3c5fb553f6978/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:19.871036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a50779-461f-450e-a7d0-d8da1a860f91 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:19.882490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a50779-461f-450e-a7d0-d8da1a860f91 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:19.954847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965c7d8-3b09-420d-ab13-7782d3825987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:19.955465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965c7d8-3b09-420d-ab13-7782d3825987 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:19.955855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965c7d8-3b09-420d-ab13-7782d3825987 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:19.956139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965c7d8-3b09-420d-ab13-7782d3825987 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:19.956782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965c7d8-3b09-420d-ab13-7782d3825987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:19.958422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965c7d8-3b09-420d-ab13-7782d3825987 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:19.968795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1165/2351] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:54:19.973217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee89532-fa69-42e6-901e-7d4968e163fc admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:19.984278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee89532-fa69-42e6-901e-7d4968e163fc admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:19.989254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03390195-a628-4b1a-9a92-dc99f192b212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:19.989991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390195-a628-4b1a-9a92-dc99f192b212 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:19.990306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390195-a628-4b1a-9a92-dc99f192b212 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:19.990511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390195-a628-4b1a-9a92-dc99f192b212 admin admin] PATH_INFO: `/v3/projects/aac7e6ed26fa439bb2d44d462b70c6d9/users/72f7e33f9a06403ab2c3c5fb553f6978/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:19.992439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03390195-a628-4b1a-9a92-dc99f192b212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac7e6ed26fa439bb2d44d462b70c6d9, user_id=72f7e33f9a06403ab2c3c5fb553f6978, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:19.995287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03390195-a628-4b1a-9a92-dc99f192b212 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:20.009591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1187/2352] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:54:19 2022] PUT /identity/v3/projects/aac7e6ed26fa439bb2d44d462b70c6d9/users/72f7e33f9a06403ab2c3c5fb553f6978/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:20.267999 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965c7d8-3b09-420d-ab13-7782d3825987 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:20.279382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965c7d8-3b09-420d-ab13-7782d3825987 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:20.314774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39870390-bfa5-469f-988e-b90496b8b451 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:20.314961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39870390-bfa5-469f-988e-b90496b8b451 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:20.315248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39870390-bfa5-469f-988e-b90496b8b451 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:20.316587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1188/2353] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:54:20.323235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97783d2f-e6d8-4b6a-be11-e7b0ec93be5e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:20.323541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97783d2f-e6d8-4b6a-be11-e7b0ec93be5e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:20.323816 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97783d2f-e6d8-4b6a-be11-e7b0ec93be5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:20.344886 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-97783d2f-e6d8-4b6a-be11-e7b0ec93be5e None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:20.346146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-97783d2f-e6d8-4b6a-be11-e7b0ec93be5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:20.377951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:20.387643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:20.388839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:20.389444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:20.389965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] PATH_INFO: `/v3/users/ecd17ab0627e4afbb7b165d974b27759` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:20.393251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd17ab0627e4afbb7b165d974b27759)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:20.395841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:20.482250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1189/2354] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:54:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:54:20.483261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97783d2f-e6d8-4b6a-be11-e7b0ec93be5e None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:20.484953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-97783d2f-e6d8-4b6a-be11-e7b0ec93be5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:20.507921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97783d2f-e6d8-4b6a-be11-e7b0ec93be5e None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:20.513536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:20.514283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:20.514629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:20.515088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] PATH_INFO: `/v3/users/24d13f3b744542b18a056e07135d5bb7` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:20.516380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=24d13f3b744542b18a056e07135d5bb7)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:20.518467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:20.671911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:20.675818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd17ab0627e4afbb7b165d974b27759'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:20.676566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd17ab0627e4afbb7b165d974b27759'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:20.677361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd17ab0627e4afbb7b165d974b27759'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:20.702798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd17ab0627e4afbb7b165d974b27759'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:20.712633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd17ab0627e4afbb7b165d974b27759'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:20.760153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24d13f3b744542b18a056e07135d5bb7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:20.761372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24d13f3b744542b18a056e07135d5bb7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:20.763650 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24d13f3b744542b18a056e07135d5bb7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:20.783647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1166/2355] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:54:20 2022] DELETE /identity/v3/users/ecd17ab0627e4afbb7b165d974b27759 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:20.786605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:20.791243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:20.799649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24d13f3b744542b18a056e07135d5bb7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:20.831379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-78c8d023-0c85-4799-8071-e5a5b43e9d4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24d13f3b744542b18a056e07135d5bb7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:20.887114 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a90c0-8c46-4509-bb40-20863f3c841b admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:20.891058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f499c42-8e0c-424e-8bde-2f03b53df7d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d10585480f41608bc6935e282f7e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:20.891808 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f499c42-8e0c-424e-8bde-2f03b53df7d8 service glance] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:20.892257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f499c42-8e0c-424e-8bde-2f03b53df7d8 service glance] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:20.892770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f499c42-8e0c-424e-8bde-2f03b53df7d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:20.894673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5f499c42-8e0c-424e-8bde-2f03b53df7d8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:20.920969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1190/2356] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:54:20 2022] DELETE /identity/v3/users/24d13f3b744542b18a056e07135d5bb7 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:20.922541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe34e6-6e97-4d8f-a455-61a2f603c827 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:20.922718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe34e6-6e97-4d8f-a455-61a2f603c827 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:20.923337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe34e6-6e97-4d8f-a455-61a2f603c827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:20.975860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-06fe34e6-6e97-4d8f-a455-61a2f603c827 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:20.977148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-06fe34e6-6e97-4d8f-a455-61a2f603c827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:21.015872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f499c42-8e0c-424e-8bde-2f03b53df7d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:21.017806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f499c42-8e0c-424e-8bde-2f03b53df7d8 service glance] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:21.035687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1167/2357] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:21.037325 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ce33e-3896-4436-a669-d219e5cc4258 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:21.037996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ce33e-3896-4436-a669-d219e5cc4258 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:21.038366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ce33e-3896-4436-a669-d219e5cc4258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:21.068602 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-055ce33e-3896-4436-a669-d219e5cc4258 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:21.070341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-055ce33e-3896-4436-a669-d219e5cc4258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:21.107168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1191/2358] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:21.118047 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe34e6-6e97-4d8f-a455-61a2f603c827 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:21.119810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-06fe34e6-6e97-4d8f-a455-61a2f603c827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:21.166017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:21.178956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1168/2359] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:21.189362 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ce33e-3896-4436-a669-d219e5cc4258 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:21.194271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:21.195556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe34e6-6e97-4d8f-a455-61a2f603c827 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:21.201464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ce33e-3896-4436-a669-d219e5cc4258 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:21.208091 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ec5a0-baae-4267-8dba-41d78a54e79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:21.209433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ec5a0-baae-4267-8dba-41d78a54e79a service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:21.210001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ec5a0-baae-4267-8dba-41d78a54e79a service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:21.211384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22084dba-ce60-48bd-978e-77487ac536a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:21.211549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ec5a0-baae-4267-8dba-41d78a54e79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:21.217525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22084dba-ce60-48bd-978e-77487ac536a0 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:21.218223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22084dba-ce60-48bd-978e-77487ac536a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:21.218512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22084dba-ce60-48bd-978e-77487ac536a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:21.226678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ec5a0-baae-4267-8dba-41d78a54e79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:21.228515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ec5a0-baae-4267-8dba-41d78a54e79a service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:21.237154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22084dba-ce60-48bd-978e-77487ac536a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:21.239628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22084dba-ce60-48bd-978e-77487ac536a0 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:21.253575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1192/2361] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:54:21.257063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1169/2361] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:54:21.334535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea9b54-ce2b-4704-b135-1099de22d4d8 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:21.334806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea9b54-ce2b-4704-b135-1099de22d4d8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:21.335151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea9b54-ce2b-4704-b135-1099de22d4d8 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:21.336728 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1193/2362] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:54:21.345415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98514367-ad65-44db-8d50-1d0f91270cc1 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:21.349727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98514367-ad65-44db-8d50-1d0f91270cc1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:21.349727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98514367-ad65-44db-8d50-1d0f91270cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:21.377130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-98514367-ad65-44db-8d50-1d0f91270cc1 None None] MFA Rules not processed for user `bcf30ef695154d3a97d7e589aca21233`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:21.378517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-98514367-ad65-44db-8d50-1d0f91270cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:21.386327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1170/2363] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:21.405274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea9b54-ce2b-4704-b135-1099de22d4d8 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:21.422013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea9b54-ce2b-4704-b135-1099de22d4d8 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:21.426288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216887a3-070a-4b1a-8c50-f3fe2ac4eba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:21.426978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216887a3-070a-4b1a-8c50-f3fe2ac4eba9 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:21.427356 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216887a3-070a-4b1a-8c50-f3fe2ac4eba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:21.427698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216887a3-070a-4b1a-8c50-f3fe2ac4eba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:21.436509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216887a3-070a-4b1a-8c50-f3fe2ac4eba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:21.438287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216887a3-070a-4b1a-8c50-f3fe2ac4eba9 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:21.447157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1194/2364] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:54:21.557826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63ba6c-32e8-46b5-a095-62095b369c9f None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:21.558177 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63ba6c-32e8-46b5-a095-62095b369c9f None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:21.558457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63ba6c-32e8-46b5-a095-62095b369c9f None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:21.559824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1171/2365] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:54:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 18:54:21.573473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa33e7-298c-4f88-9f00-2889cb744880 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:21.573777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa33e7-298c-4f88-9f00-2889cb744880 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:21.574593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa33e7-298c-4f88-9f00-2889cb744880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:21.614274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c3aa33e7-298c-4f88-9f00-2889cb744880 None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:21.625957 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c3aa33e7-298c-4f88-9f00-2889cb744880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:21.758745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1195/2366] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:54:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:54:21.770561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63ba6c-32e8-46b5-a095-62095b369c9f None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:21.785597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63ba6c-32e8-46b5-a095-62095b369c9f None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:21.789713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808fc6b-fc1e-42a0-b72d-eb0ed110c618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:21.790635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808fc6b-fc1e-42a0-b72d-eb0ed110c618 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:21.790635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808fc6b-fc1e-42a0-b72d-eb0ed110c618 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:21.790936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808fc6b-fc1e-42a0-b72d-eb0ed110c618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:21.804509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808fc6b-fc1e-42a0-b72d-eb0ed110c618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:21.807185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808fc6b-fc1e-42a0-b72d-eb0ed110c618 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:21.823708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1172/2367] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:22.060513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa33e7-298c-4f88-9f00-2889cb744880 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:22.063651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c3aa33e7-298c-4f88-9f00-2889cb744880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:22.077296 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808fc6b-fc1e-42a0-b72d-eb0ed110c618 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:22.082595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1808fc6b-fc1e-42a0-b72d-eb0ed110c618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:22.114287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa33e7-298c-4f88-9f00-2889cb744880 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:22.119644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808fc6b-fc1e-42a0-b72d-eb0ed110c618 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:22.134678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:22.136694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:22.138206 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:22.138206 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:22.138434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:22.138434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:22.139005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] PATH_INFO: `/v3/projects/46c3deb0f3f64116ba78bb2e71b3bc7c` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:22.140711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] PATH_INFO: `/v3/projects/8fcee5c6e95c467683a10d652bff9470` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:22.146198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46c3deb0f3f64116ba78bb2e71b3bc7c)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:22.147978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:22.151000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fcee5c6e95c467683a10d652bff9470)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:22.152934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:22.340932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46c3deb0f3f64116ba78bb2e71b3bc7c', 'request_id': 'req-3a37389c-5e17-48c5-a6f5-5c932741c258'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:22.352381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46c3deb0f3f64116ba78bb2e71b3bc7c', 'request_id': 'req-3a37389c-5e17-48c5-a6f5-5c932741c258'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:22.383517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46c3deb0f3f64116ba78bb2e71b3bc7c', 'request_id': 'req-3a37389c-5e17-48c5-a6f5-5c932741c258'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:22.401501 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fcee5c6e95c467683a10d652bff9470', 'request_id': 'req-8918c7a6-2ae0-4580-80e7-8e3edd884465'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:22.413695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fcee5c6e95c467683a10d652bff9470', 'request_id': 'req-8918c7a6-2ae0-4580-80e7-8e3edd884465'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:22.609353 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fcee5c6e95c467683a10d652bff9470', 'request_id': 'req-8918c7a6-2ae0-4580-80e7-8e3edd884465'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:22.643695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] The token cache is being invalidate because project 8fcee5c6e95c467683a10d652bff9470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:54:22.643890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:22.644656 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-8918c7a6-2ae0-4580-80e7-8e3edd884465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:22.646179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1196/2368] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:54:22 2022] DELETE /identity/v3/projects/8fcee5c6e95c467683a10d652bff9470 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:22.651553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502b3d0-db4c-44bb-9f1b-1a6ead62cc10 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:22.651839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502b3d0-db4c-44bb-9f1b-1a6ead62cc10 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:22.652118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502b3d0-db4c-44bb-9f1b-1a6ead62cc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:22.678357 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-6502b3d0-db4c-44bb-9f1b-1a6ead62cc10 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:22.679438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6502b3d0-db4c-44bb-9f1b-1a6ead62cc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:22.700449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] The token cache is being invalidate because project 46c3deb0f3f64116ba78bb2e71b3bc7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:54:22.701320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:22.702951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3a37389c-5e17-48c5-a6f5-5c932741c258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:22.707256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1173/2369] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:54:22 2022] DELETE /identity/v3/projects/46c3deb0f3f64116ba78bb2e71b3bc7c => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:54:22.713494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623ac6f-f134-4913-921a-783675fddcbb None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:22.716527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623ac6f-f134-4913-921a-783675fddcbb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:22.717710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623ac6f-f134-4913-921a-783675fddcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:22.732028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:22.760893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a623ac6f-f134-4913-921a-783675fddcbb None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:22.763098 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a623ac6f-f134-4913-921a-783675fddcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:22.793523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1197/2370] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:22.803450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89f11e-b85b-4625-9554-18fa2c966db8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:22.809685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89f11e-b85b-4625-9554-18fa2c966db8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:22.809685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89f11e-b85b-4625-9554-18fa2c966db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:22.835263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-cc89f11e-b85b-4625-9554-18fa2c966db8 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:22.836614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-cc89f11e-b85b-4625-9554-18fa2c966db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:22.846972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1198/2371] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:54:22.853058 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89f11e-b85b-4625-9554-18fa2c966db8 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:22.877482 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89f11e-b85b-4625-9554-18fa2c966db8 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:22.893330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfa8ef-1514-4933-a658-1121b4f1c652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:22.894347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfa8ef-1514-4933-a658-1121b4f1c652 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:22.894816 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfa8ef-1514-4933-a658-1121b4f1c652 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:22.895449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfa8ef-1514-4933-a658-1121b4f1c652 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:22.896108 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1174/2372] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:22.896510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfa8ef-1514-4933-a658-1121b4f1c652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:54:22.896997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfa8ef-1514-4933-a658-1121b4f1c652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:22.901364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfa8ef-1514-4933-a658-1121b4f1c652 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:22.915819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1199/2373] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:54:22.926202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623ac6f-f134-4913-921a-783675fddcbb None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:22.948012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623ac6f-f134-4913-921a-783675fddcbb None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:22.972063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ccd39-406a-4c3e-98cd-17634cbd4577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:22.973410 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ccd39-406a-4c3e-98cd-17634cbd4577 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:22.974175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ccd39-406a-4c3e-98cd-17634cbd4577 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:22.974638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ccd39-406a-4c3e-98cd-17634cbd4577 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:22.983578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ccd39-406a-4c3e-98cd-17634cbd4577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:22.986663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca553d1-d518-4364-b70a-c21f816d9137 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:22.987218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca553d1-d518-4364-b70a-c21f816d9137 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:22.987488 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca553d1-d518-4364-b70a-c21f816d9137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:22.992710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ccd39-406a-4c3e-98cd-17634cbd4577 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.015484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-eca553d1-d518-4364-b70a-c21f816d9137 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:23.016727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-eca553d1-d518-4364-b70a-c21f816d9137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:23.049759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1200/2374] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:54:23.053620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca553d1-d518-4364-b70a-c21f816d9137 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:23.067496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca553d1-d518-4364-b70a-c21f816d9137 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:23.075239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea0ed8-1d4e-4c06-80d0-227bd075dff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:23.078673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea0ed8-1d4e-4c06-80d0-227bd075dff4 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.078673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea0ed8-1d4e-4c06-80d0-227bd075dff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.078673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea0ed8-1d4e-4c06-80d0-227bd075dff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.079239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea0ed8-1d4e-4c06-80d0-227bd075dff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:54:23.079615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea0ed8-1d4e-4c06-80d0-227bd075dff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:23.081399 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea0ed8-1d4e-4c06-80d0-227bd075dff4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.107377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1201/2375] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:54:23.109013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea0ed8-1d4e-4c06-80d0-227bd075dff4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:23.131300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea0ed8-1d4e-4c06-80d0-227bd075dff4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:23.144694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56bd37-be30-4511-9f15-6e8d6198603b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:23.147862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56bd37-be30-4511-9f15-6e8d6198603b admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.148151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56bd37-be30-4511-9f15-6e8d6198603b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.148946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56bd37-be30-4511-9f15-6e8d6198603b admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.150273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56bd37-be30-4511-9f15-6e8d6198603b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:23.152343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56bd37-be30-4511-9f15-6e8d6198603b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.167329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1175/2376] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:54:22 2022] POST /identity/v3/projects => generated 399 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:54:23.171796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ccd39-406a-4c3e-98cd-17634cbd4577 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:23.215927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:23.261967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ccd39-406a-4c3e-98cd-17634cbd4577 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:23.275301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5487b3-05eb-40d0-b774-7d3628780781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:23.275545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5487b3-05eb-40d0-b774-7d3628780781 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.276009 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5487b3-05eb-40d0-b774-7d3628780781 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.276435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5487b3-05eb-40d0-b774-7d3628780781 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.277264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5487b3-05eb-40d0-b774-7d3628780781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:23.278025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1202/2377] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:54:23 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:54:23.279577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5487b3-05eb-40d0-b774-7d3628780781 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.283532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56bd37-be30-4511-9f15-6e8d6198603b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:23.410634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56bd37-be30-4511-9f15-6e8d6198603b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:23.419131 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795ae06-fe06-48dd-b0c4-d53d3d78e46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:23.419913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795ae06-fe06-48dd-b0c4-d53d3d78e46b admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.420298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795ae06-fe06-48dd-b0c4-d53d3d78e46b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.420775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795ae06-fe06-48dd-b0c4-d53d3d78e46b admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.421889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795ae06-fe06-48dd-b0c4-d53d3d78e46b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:23.424560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795ae06-fe06-48dd-b0c4-d53d3d78e46b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.519031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1176/2378] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:54:23 2022] POST /identity/v3/users => generated 300 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:54:23.520586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5487b3-05eb-40d0-b774-7d3628780781 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:23.531018 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1203/2379] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:54:23 2022] POST /identity/v3/users => generated 284 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:54:23.536422 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795ae06-fe06-48dd-b0c4-d53d3d78e46b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:23.569101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795ae06-fe06-48dd-b0c4-d53d3d78e46b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:23.569502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5487b3-05eb-40d0-b774-7d3628780781 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:23.595423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d8291-4def-4ce3-8d4f-5f39a2d31b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:23.597513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d8291-4def-4ce3-8d4f-5f39a2d31b30 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.599298 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d8291-4def-4ce3-8d4f-5f39a2d31b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.600442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d909e4c-a770-44ab-9b33-f78ce48401fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:23.601531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d8291-4def-4ce3-8d4f-5f39a2d31b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.603605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d909e4c-a770-44ab-9b33-f78ce48401fa admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.604437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d909e4c-a770-44ab-9b33-f78ce48401fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.605517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d8291-4def-4ce3-8d4f-5f39a2d31b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:23.606459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d909e4c-a770-44ab-9b33-f78ce48401fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.609528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d909e4c-a770-44ab-9b33-f78ce48401fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:23.613781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d8291-4def-4ce3-8d4f-5f39a2d31b30 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.615992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d909e4c-a770-44ab-9b33-f78ce48401fa admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.660384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1204/2380] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:54:23.661961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d909e4c-a770-44ab-9b33-f78ce48401fa admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:23.672095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1177/2381] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:54:23.675533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d8291-4def-4ce3-8d4f-5f39a2d31b30 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:23.699908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d8291-4def-4ce3-8d4f-5f39a2d31b30 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:23.702856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d909e4c-a770-44ab-9b33-f78ce48401fa admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:23.706726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080f7a8-793e-47d3-a83f-9c57c1fcfb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:23.707704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080f7a8-793e-47d3-a83f-9c57c1fcfb4c admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.707879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba691e-16f7-473e-bac2-881e515c5305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:23.708347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080f7a8-793e-47d3-a83f-9c57c1fcfb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.708503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba691e-16f7-473e-bac2-881e515c5305 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.708768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba691e-16f7-473e-bac2-881e515c5305 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.709408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba691e-16f7-473e-bac2-881e515c5305 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.711045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba691e-16f7-473e-bac2-881e515c5305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:23.711407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080f7a8-793e-47d3-a83f-9c57c1fcfb4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.713215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080f7a8-793e-47d3-a83f-9c57c1fcfb4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:23.714582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba691e-16f7-473e-bac2-881e515c5305 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.716473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080f7a8-793e-47d3-a83f-9c57c1fcfb4c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.735528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1178/2382] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:54:23.737774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080f7a8-793e-47d3-a83f-9c57c1fcfb4c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:23.756099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1205/2383] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:54:23.758585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba691e-16f7-473e-bac2-881e515c5305 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:23.759930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080f7a8-793e-47d3-a83f-9c57c1fcfb4c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:23.769792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba691e-16f7-473e-bac2-881e515c5305 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:23.775227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641a8df-ad1f-4723-91fa-1ea5e115c317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:23.776450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641a8df-ad1f-4723-91fa-1ea5e115c317 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.776634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a208a1d-fe44-4acf-a80a-686f976fec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:23.777365 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a208a1d-fe44-4acf-a80a-686f976fec42 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.777635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641a8df-ad1f-4723-91fa-1ea5e115c317 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.777815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a208a1d-fe44-4acf-a80a-686f976fec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.778103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a208a1d-fe44-4acf-a80a-686f976fec42 admin admin] PATH_INFO: `/v3/projects/bb654df7190d4f48b9db703b58b37a81/users/5ac4c7e90d7d4bb7a2c4959f7562ef41/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.778669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641a8df-ad1f-4723-91fa-1ea5e115c317 admin admin] PATH_INFO: `/v3/projects/ac89c95e6d14478e98e9370d30b8782b/users/8a6e9564c42740baa96871b81ef938c4/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.805004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641a8df-ad1f-4723-91fa-1ea5e115c317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac89c95e6d14478e98e9370d30b8782b, user_id=8a6e9564c42740baa96871b81ef938c4, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:23.806511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a208a1d-fe44-4acf-a80a-686f976fec42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb654df7190d4f48b9db703b58b37a81, user_id=5ac4c7e90d7d4bb7a2c4959f7562ef41, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:23.809777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a208a1d-fe44-4acf-a80a-686f976fec42 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.812371 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641a8df-ad1f-4723-91fa-1ea5e115c317 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:23.937112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1206/2385] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:54:23 2022] PUT /identity/v3/projects/bb654df7190d4f48b9db703b58b37a81/users/5ac4c7e90d7d4bb7a2c4959f7562ef41/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:23.937819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1179/2385] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:54:23 2022] PUT /identity/v3/projects/ac89c95e6d14478e98e9370d30b8782b/users/8a6e9564c42740baa96871b81ef938c4/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:23.939417 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78a6fa-7f96-4c09-81d5-ae563e50c7fb None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.940013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78a6fa-7f96-4c09-81d5-ae563e50c7fb None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.940419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78a6fa-7f96-4c09-81d5-ae563e50c7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.942010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857d3b3-e01d-4b19-bb30-45511228add2 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:23.944426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857d3b3-e01d-4b19-bb30-45511228add2 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:23.944907 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857d3b3-e01d-4b19-bb30-45511228add2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:23.988085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1f78a6fa-7f96-4c09-81d5-ae563e50c7fb None None] MFA Rules not processed for user `8a6e9564c42740baa96871b81ef938c4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:23.989615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1f78a6fa-7f96-4c09-81d5-ae563e50c7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:23.997726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:24.031050 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-3857d3b3-e01d-4b19-bb30-45511228add2 None None] MFA Rules not processed for user `5ac4c7e90d7d4bb7a2c4959f7562ef41`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:24.036594 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3857d3b3-e01d-4b19-bb30-45511228add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:24.114197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1207/2386] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:24.119416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78a6fa-7f96-4c09-81d5-ae563e50c7fb None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:24.125714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1f78a6fa-7f96-4c09-81d5-ae563e50c7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:24.278781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1180/2387] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:24.282330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857d3b3-e01d-4b19-bb30-45511228add2 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:24.283261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78a6fa-7f96-4c09-81d5-ae563e50c7fb None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:24.286025 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3857d3b3-e01d-4b19-bb30-45511228add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:24.296767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168668a7-27f5-4d7f-9bda-df429eb63426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:24.299285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168668a7-27f5-4d7f-9bda-df429eb63426 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:24.299816 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168668a7-27f5-4d7f-9bda-df429eb63426 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:24.301346 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168668a7-27f5-4d7f-9bda-df429eb63426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:24.326787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168668a7-27f5-4d7f-9bda-df429eb63426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:24.330669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168668a7-27f5-4d7f-9bda-df429eb63426 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:24.342106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1208/2388] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:24 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:24.414384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857d3b3-e01d-4b19-bb30-45511228add2 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:24.424878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5863f7a-a085-4648-a484-09f46416cc55 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d10585480f41608bc6935e282f7e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:24.425857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5863f7a-a085-4648-a484-09f46416cc55 service glance] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:24.426145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5863f7a-a085-4648-a484-09f46416cc55 service glance] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:24.426422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5863f7a-a085-4648-a484-09f46416cc55 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:24.441255 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5863f7a-a085-4648-a484-09f46416cc55 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:24.443024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5863f7a-a085-4648-a484-09f46416cc55 service glance] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:24.460040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1181/2389] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:27.727136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba2587-5cbe-4f70-b2c2-fe450cc5cc44 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:27.727136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba2587-5cbe-4f70-b2c2-fe450cc5cc44 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:27.727136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba2587-5cbe-4f70-b2c2-fe450cc5cc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:27.763332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-25ba2587-5cbe-4f70-b2c2-fe450cc5cc44 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:27.766224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-25ba2587-5cbe-4f70-b2c2-fe450cc5cc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:27.866242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1209/2390] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:54:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:27.871287 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5863f7a-a085-4648-a484-09f46416cc55 service glance] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:27.872849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f5863f7a-a085-4648-a484-09f46416cc55 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:27.887161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:28.006091 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5863f7a-a085-4648-a484-09f46416cc55 service glance] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:28.013477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc234c64-31b9-49b7-bc1e-aa99b1343f7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:28.015294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc234c64-31b9-49b7-bc1e-aa99b1343f7d service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:28.015785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc234c64-31b9-49b7-bc1e-aa99b1343f7d service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:28.016065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc234c64-31b9-49b7-bc1e-aa99b1343f7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:28.028490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc234c64-31b9-49b7-bc1e-aa99b1343f7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:28.030356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc234c64-31b9-49b7-bc1e-aa99b1343f7d service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:28.052803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1182/2391] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:54:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 18:54:28.940104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97107b-7203-45ff-9888-74271a517400 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:28.940634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97107b-7203-45ff-9888-74271a517400 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:28.940634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97107b-7203-45ff-9888-74271a517400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:28.976656 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4e97107b-7203-45ff-9888-74271a517400 None None] MFA Rules not processed for user `72f7e33f9a06403ab2c3c5fb553f6978`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:28.979398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4e97107b-7203-45ff-9888-74271a517400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:29.034729 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:29.075167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1210/2392] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:34.253441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc234c64-31b9-49b7-bc1e-aa99b1343f7d service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:34.356422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc234c64-31b9-49b7-bc1e-aa99b1343f7d service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:34.361505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:34.362484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:34.362739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:34.363012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] PATH_INFO: `/v3/users/8a6e9564c42740baa96871b81ef938c4` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:34.364391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a6e9564c42740baa96871b81ef938c4)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:34.366457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:34.640307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a6e9564c42740baa96871b81ef938c4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:34.640974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a6e9564c42740baa96871b81ef938c4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:34.641627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a6e9564c42740baa96871b81ef938c4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:34.658138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a6e9564c42740baa96871b81ef938c4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:34.666744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a6e9564c42740baa96871b81ef938c4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:34.756619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1183/2393] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:54:34 2022] DELETE /identity/v3/users/8a6e9564c42740baa96871b81ef938c4 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:34.761944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5dbd6-3c1a-4189-9811-8dc32f6bebb4 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:34.762513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5dbd6-3c1a-4189-9811-8dc32f6bebb4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:34.762513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5dbd6-3c1a-4189-9811-8dc32f6bebb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:34.793708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-29c5dbd6-3c1a-4189-9811-8dc32f6bebb4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:34.795248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-29c5dbd6-3c1a-4189-9811-8dc32f6bebb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:34.872684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1211/2394] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:34.943671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:34.945677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:34.971975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:35.016150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754e2b0-5eb0-4d99-8617-da34c7c56377 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:35.019806 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700c951-344d-4848-8de1-7aea105b1e6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:35.020440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700c951-344d-4848-8de1-7aea105b1e6b service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:35.020719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700c951-344d-4848-8de1-7aea105b1e6b service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:35.021005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700c951-344d-4848-8de1-7aea105b1e6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:35.032135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700c951-344d-4848-8de1-7aea105b1e6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:35.033631 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700c951-344d-4848-8de1-7aea105b1e6b service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:35.045109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1184/2395] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:35.611717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5dbd6-3c1a-4189-9811-8dc32f6bebb4 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:35.613415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-29c5dbd6-3c1a-4189-9811-8dc32f6bebb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:35.625725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5dbd6-3c1a-4189-9811-8dc32f6bebb4 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:35.630933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:35.631436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:35.631750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:35.632033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] PATH_INFO: `/v3/projects/ac89c95e6d14478e98e9370d30b8782b` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:35.633210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac89c95e6d14478e98e9370d30b8782b)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:35.635916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:35.778874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac89c95e6d14478e98e9370d30b8782b', 'request_id': 'req-ae058235-deb8-4034-bfe0-c04f77801db5'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:35.785582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac89c95e6d14478e98e9370d30b8782b', 'request_id': 'req-ae058235-deb8-4034-bfe0-c04f77801db5'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:35.859695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac89c95e6d14478e98e9370d30b8782b', 'request_id': 'req-ae058235-deb8-4034-bfe0-c04f77801db5'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:35.875117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] The token cache is being invalidate because project ac89c95e6d14478e98e9370d30b8782b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:54:35.877096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:35.877096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ae058235-deb8-4034-bfe0-c04f77801db5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:54:35.878850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1212/2396] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:54:35 2022] DELETE /identity/v3/projects/ac89c95e6d14478e98e9370d30b8782b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:54:37.764134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700c951-344d-4848-8de1-7aea105b1e6b service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:37.766206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2700c951-344d-4848-8de1-7aea105b1e6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:37.824605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700c951-344d-4848-8de1-7aea105b1e6b service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:37.878639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c397434-3d8f-4255-a777-59a7de7f0315 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d10585480f41608bc6935e282f7e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:37.879727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c397434-3d8f-4255-a777-59a7de7f0315 service glance] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:37.884061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c397434-3d8f-4255-a777-59a7de7f0315 service glance] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:37.884061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c397434-3d8f-4255-a777-59a7de7f0315 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:37.886205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6c397434-3d8f-4255-a777-59a7de7f0315 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:37.957837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c397434-3d8f-4255-a777-59a7de7f0315 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:37.959197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c397434-3d8f-4255-a777-59a7de7f0315 service glance] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:37.969351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1185/2397] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:38.249906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd9e8a-e8e1-49a3-9ce3-fd5c49ec3065 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:38.250387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd9e8a-e8e1-49a3-9ce3-fd5c49ec3065 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:38.250480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd9e8a-e8e1-49a3-9ce3-fd5c49ec3065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:38.271220 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-74cd9e8a-e8e1-49a3-9ce3-fd5c49ec3065 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:38.272470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-74cd9e8a-e8e1-49a3-9ce3-fd5c49ec3065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:38.345024 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1213/2398] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:54:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:54:38.357849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c397434-3d8f-4255-a777-59a7de7f0315 service glance] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:38.359502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6c397434-3d8f-4255-a777-59a7de7f0315 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:38.395233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:38.433210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c397434-3d8f-4255-a777-59a7de7f0315 service glance] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:38.441014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1771dff-cc94-49be-bbbe-70659d39cde0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:38.441717 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771dff-cc94-49be-bbbe-70659d39cde0 service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:38.441995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771dff-cc94-49be-bbbe-70659d39cde0 service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:38.442373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771dff-cc94-49be-bbbe-70659d39cde0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:38.455597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1771dff-cc94-49be-bbbe-70659d39cde0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:38.457654 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1771dff-cc94-49be-bbbe-70659d39cde0 service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:38.469755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1186/2399] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:54:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 18:54:41.561363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd9e8a-e8e1-49a3-9ce3-fd5c49ec3065 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:54:41.563152 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-74cd9e8a-e8e1-49a3-9ce3-fd5c49ec3065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:41.574540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:41.643767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd9e8a-e8e1-49a3-9ce3-fd5c49ec3065 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:54:41.649174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00cb0e-2f60-431c-a8c2-80d2bb997b7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:54:41.649858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00cb0e-2f60-431c-a8c2-80d2bb997b7b service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:41.650141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00cb0e-2f60-431c-a8c2-80d2bb997b7b service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:41.650421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00cb0e-2f60-431c-a8c2-80d2bb997b7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:41.652129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-fc00cb0e-2f60-431c-a8c2-80d2bb997b7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:41.665076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00cb0e-2f60-431c-a8c2-80d2bb997b7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:54:41.666672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00cb0e-2f60-431c-a8c2-80d2bb997b7b service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:54:41.676647 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1214/2400] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:54:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:54:48.468024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=19 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:49.183465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ee3cd-b535-4e23-b0a5-a6bd28a2ed06 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:54:49.183755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ee3cd-b535-4e23-b0a5-a6bd28a2ed06 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:54:49.184029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ee3cd-b535-4e23-b0a5-a6bd28a2ed06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:54:49.203815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-4a1ee3cd-b535-4e23-b0a5-a6bd28a2ed06 None None] MFA Rules not processed for user `f7555f03f8e34998ab1890279bf9b5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:54:49.206215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4a1ee3cd-b535-4e23-b0a5-a6bd28a2ed06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:54:49.211924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1187/2401] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:54:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:54:51.710943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=28 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:54:59.198443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:55:03.253631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932b4d-03c1-4a78-818c-d71479b53562 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:03.254115 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932b4d-03c1-4a78-818c-d71479b53562 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:03.261151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932b4d-03c1-4a78-818c-d71479b53562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:03.274316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-0b932b4d-03c1-4a78-818c-d71479b53562 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:55:03.275561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0b932b4d-03c1-4a78-818c-d71479b53562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:03.283595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1215/2402] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 18:55:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:55:03.308996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ee3cd-b535-4e23-b0a5-a6bd28a2ed06 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:55:03.335596 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ee3cd-b535-4e23-b0a5-a6bd28a2ed06 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:55:03.342461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ee5c8-cbfd-45b1-8672-84b2a565a266 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:55:03.343212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ee5c8-cbfd-45b1-8672-84b2a565a266 service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:03.343368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ee5c8-cbfd-45b1-8672-84b2a565a266 service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:03.343759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ee5c8-cbfd-45b1-8672-84b2a565a266 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:03.353880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ee5c8-cbfd-45b1-8672-84b2a565a266 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:55:03.358705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ee5c8-cbfd-45b1-8672-84b2a565a266 service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:55:03.364472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1188/2403] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:55:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 18:55:03.736539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b3002-bea4-4def-9265-d05a7ca95359 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:03.737031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b3002-bea4-4def-9265-d05a7ca95359 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:03.737402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b3002-bea4-4def-9265-d05a7ca95359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:03.757203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1c2b3002-bea4-4def-9265-d05a7ca95359 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:55:03.758426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1c2b3002-bea4-4def-9265-d05a7ca95359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:03.766954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1216/2404] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:55:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:55:03.781670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ee5c8-cbfd-45b1-8672-84b2a565a266 service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:55:03.794991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ee5c8-cbfd-45b1-8672-84b2a565a266 service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:55:03.799858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e9033-08b2-4202-abd3-f4bf8472fdf4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:55:03.800459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e9033-08b2-4202-abd3-f4bf8472fdf4 service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:03.800799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e9033-08b2-4202-abd3-f4bf8472fdf4 service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:03.801109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e9033-08b2-4202-abd3-f4bf8472fdf4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:03.809489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e9033-08b2-4202-abd3-f4bf8472fdf4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:55:03.811103 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e9033-08b2-4202-abd3-f4bf8472fdf4 service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:55:03.819451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1189/2405] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:55:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 18:55:06.353926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511ee2c-2ac0-4304-890a-936fb414178d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:06.354544 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511ee2c-2ac0-4304-890a-936fb414178d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:06.355271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511ee2c-2ac0-4304-890a-936fb414178d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:06.381398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2511ee2c-2ac0-4304-890a-936fb414178d None None] MFA Rules not processed for user `8da410d2ea0740dc8eeea87244fc1408`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:55:06.382719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2511ee2c-2ac0-4304-890a-936fb414178d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:06.456640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1217/2406] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:55:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:55:06.782253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e9033-08b2-4202-abd3-f4bf8472fdf4 service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:55:06.784994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6a3e9033-08b2-4202-abd3-f4bf8472fdf4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:06.927903 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e9033-08b2-4202-abd3-f4bf8472fdf4 service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:55:06.941581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:55:06.941581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:06.941581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:06.941581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] PATH_INFO: `/v3/users/5ac4c7e90d7d4bb7a2c4959f7562ef41` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:06.942012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac4c7e90d7d4bb7a2c4959f7562ef41)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:55:06.942012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:55:07.130424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac4c7e90d7d4bb7a2c4959f7562ef41'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:55:07.131200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac4c7e90d7d4bb7a2c4959f7562ef41'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:55:07.132212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac4c7e90d7d4bb7a2c4959f7562ef41'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:55:07.156339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac4c7e90d7d4bb7a2c4959f7562ef41'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:55:07.170000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac4c7e90d7d4bb7a2c4959f7562ef41'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:55:07.233093 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1190/2407] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:55:06 2022] DELETE /identity/v3/users/5ac4c7e90d7d4bb7a2c4959f7562ef41 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:55:07.236505 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469bfe8-b015-4376-9192-a658c78cccc6 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:07.236799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469bfe8-b015-4376-9192-a658c78cccc6 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:07.237289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469bfe8-b015-4376-9192-a658c78cccc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:07.257360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-6469bfe8-b015-4376-9192-a658c78cccc6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:55:07.258824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6469bfe8-b015-4376-9192-a658c78cccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:07.351046 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1218/2408] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:55:07.369983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:55:07.374072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:07.551532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9832abb-59bc-43a1-ab58-786dc25c7a06 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:55:07.555475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66bd59-a98c-46ad-a0fd-1b8a22e6a8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:55:07.556252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66bd59-a98c-46ad-a0fd-1b8a22e6a8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:07.556677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66bd59-a98c-46ad-a0fd-1b8a22e6a8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:07.556993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66bd59-a98c-46ad-a0fd-1b8a22e6a8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:07.570974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:55:07.572285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66bd59-a98c-46ad-a0fd-1b8a22e6a8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:55:07.574708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66bd59-a98c-46ad-a0fd-1b8a22e6a8a0 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:55:07.589724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1191/2409] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:55:08.178459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469bfe8-b015-4376-9192-a658c78cccc6 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:55:08.178459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6469bfe8-b015-4376-9192-a658c78cccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:08.200491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469bfe8-b015-4376-9192-a658c78cccc6 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:55:08.207953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:55:08.208951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:08.209391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:08.209758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] PATH_INFO: `/v3/projects/bb654df7190d4f48b9db703b58b37a81` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:08.219593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb654df7190d4f48b9db703b58b37a81)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:55:08.221709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:55:08.315332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb654df7190d4f48b9db703b58b37a81', 'request_id': 'req-ddbd0530-40fc-46f8-96c7-82e66c2808ac'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:55:08.322250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb654df7190d4f48b9db703b58b37a81', 'request_id': 'req-ddbd0530-40fc-46f8-96c7-82e66c2808ac'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:55:08.413518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb654df7190d4f48b9db703b58b37a81', 'request_id': 'req-ddbd0530-40fc-46f8-96c7-82e66c2808ac'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:55:08.426375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] The token cache is being invalidate because project bb654df7190d4f48b9db703b58b37a81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:55:08.426672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:55:08.427335 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-ddbd0530-40fc-46f8-96c7-82e66c2808ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:55:08.428836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1219/2410] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:55:08 2022] DELETE /identity/v3/projects/bb654df7190d4f48b9db703b58b37a81 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:55:17.670379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:55:18.423006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:55:38.468219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66bd59-a98c-46ad-a0fd-1b8a22e6a8a0 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:55:38.469902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7e66bd59-a98c-46ad-a0fd-1b8a22e6a8a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:38.481120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66bd59-a98c-46ad-a0fd-1b8a22e6a8a0 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:55:38.530370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06778477-0408-4268-be20-8e8d2ea1d9ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:55:38.531031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06778477-0408-4268-be20-8e8d2ea1d9ee service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:38.531300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06778477-0408-4268-be20-8e8d2ea1d9ee service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:38.531568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06778477-0408-4268-be20-8e8d2ea1d9ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:38.533327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-06778477-0408-4268-be20-8e8d2ea1d9ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:38.595741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06778477-0408-4268-be20-8e8d2ea1d9ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:55:38.597244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06778477-0408-4268-be20-8e8d2ea1d9ee service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:55:38.608374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1192/2411] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:55:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:55:43.431724 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10c510-288f-4d33-b856-94f6295fabec None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:43.432282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10c510-288f-4d33-b856-94f6295fabec None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:43.432385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10c510-288f-4d33-b856-94f6295fabec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:43.451490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-3e10c510-288f-4d33-b856-94f6295fabec None None] MFA Rules not processed for user `91e852d04f9f464ab8afbaeb9ff355e6`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:55:43.452678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3e10c510-288f-4d33-b856-94f6295fabec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:43.509520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1220/2412] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:55:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:55:48.603215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=50 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:55:53.502923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=25 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:55:57.928636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f02c62-6809-461a-b328-0a36ba8c93e9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:55:57.929179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f02c62-6809-461a-b328-0a36ba8c93e9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:55:57.929275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f02c62-6809-461a-b328-0a36ba8c93e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:55:57.949251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a3f02c62-6809-461a-b328-0a36ba8c93e9 None None] MFA Rules not processed for user `2b57c46bc37940718735201095453f16`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:55:57.950464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a3f02c62-6809-461a-b328-0a36ba8c93e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:55:58.005986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1193/2413] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:55:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:56:07.998507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=25 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:56:23.625890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10c510-288f-4d33-b856-94f6295fabec None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:56:23.627406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3e10c510-288f-4d33-b856-94f6295fabec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:23.640296 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10c510-288f-4d33-b856-94f6295fabec None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:56:23.643797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf29d1-9772-4d2d-9458-5a3c327d0488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:56:23.644393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf29d1-9772-4d2d-9458-5a3c327d0488 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:56:23.644674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf29d1-9772-4d2d-9458-5a3c327d0488 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:56:23.644948 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf29d1-9772-4d2d-9458-5a3c327d0488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:56:23.647016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-09bf29d1-9772-4d2d-9458-5a3c327d0488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:23.745377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf29d1-9772-4d2d-9458-5a3c327d0488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:56:23.746792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf29d1-9772-4d2d-9458-5a3c327d0488 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:56:23.758280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1221/2414] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:56:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:56:24.258582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f02c62-6809-461a-b328-0a36ba8c93e9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:56:24.268119 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f02c62-6809-461a-b328-0a36ba8c93e9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:56:24.271965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b58a4-8d81-49cd-8c90-f45433b94141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:56:24.272561 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b58a4-8d81-49cd-8c90-f45433b94141 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:56:24.272841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b58a4-8d81-49cd-8c90-f45433b94141 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:56:24.273147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b58a4-8d81-49cd-8c90-f45433b94141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:56:24.274943 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9b2b58a4-8d81-49cd-8c90-f45433b94141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:24.288771 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b58a4-8d81-49cd-8c90-f45433b94141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:56:24.290330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b58a4-8d81-49cd-8c90-f45433b94141 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:56:24.301779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1194/2415] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:56:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:56:33.190274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4c5ef-d53a-4615-afcc-5808e1f56d49 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:56:33.195741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4c5ef-d53a-4615-afcc-5808e1f56d49 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:56:33.195741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4c5ef-d53a-4615-afcc-5808e1f56d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:56:33.211328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-91e4c5ef-d53a-4615-afcc-5808e1f56d49 None None] MFA Rules not processed for user `8da410d2ea0740dc8eeea87244fc1408`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:56:33.212692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-91e4c5ef-d53a-4615-afcc-5808e1f56d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:33.266568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1222/2416] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:56:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:56:34.296672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:56:34.764229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b58a4-8d81-49cd-8c90-f45433b94141 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:56:34.765688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9b2b58a4-8d81-49cd-8c90-f45433b94141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:34.780210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b58a4-8d81-49cd-8c90-f45433b94141 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:56:34.785701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0751a-3aeb-4442-a199-cef9ef9a941e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:56:34.786293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0751a-3aeb-4442-a199-cef9ef9a941e service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:56:34.786576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0751a-3aeb-4442-a199-cef9ef9a941e service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:56:34.786848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0751a-3aeb-4442-a199-cef9ef9a941e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:56:34.788543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-07d0751a-3aeb-4442-a199-cef9ef9a941e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:34.799244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0751a-3aeb-4442-a199-cef9ef9a941e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:56:34.800818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0751a-3aeb-4442-a199-cef9ef9a941e service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:56:34.810929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1195/2417] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:56:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:56:43.263333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=59 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:56:44.806964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:56:49.764900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4c5ef-d53a-4615-afcc-5808e1f56d49 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:56:49.766486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-91e4c5ef-d53a-4615-afcc-5808e1f56d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:49.784980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4c5ef-d53a-4615-afcc-5808e1f56d49 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:56:49.788860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37619e64-c707-44cb-b498-6012f3bd852a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:56:49.789502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37619e64-c707-44cb-b498-6012f3bd852a service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:56:49.789780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37619e64-c707-44cb-b498-6012f3bd852a service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:56:49.790050 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37619e64-c707-44cb-b498-6012f3bd852a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:56:49.792467 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-37619e64-c707-44cb-b498-6012f3bd852a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:49.806722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37619e64-c707-44cb-b498-6012f3bd852a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:56:49.808330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37619e64-c707-44cb-b498-6012f3bd852a service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:56:49.819677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1223/2418] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:56:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:56:53.914905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0751a-3aeb-4442-a199-cef9ef9a941e service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:56:53.916412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-07d0751a-3aeb-4442-a199-cef9ef9a941e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:53.930972 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0751a-3aeb-4442-a199-cef9ef9a941e service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:56:53.936672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e37a77-3d83-4914-8950-944525cc0f87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:56:53.938020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e37a77-3d83-4914-8950-944525cc0f87 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:56:53.938020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e37a77-3d83-4914-8950-944525cc0f87 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:56:53.938020 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e37a77-3d83-4914-8950-944525cc0f87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:56:53.939557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d8e37a77-3d83-4914-8950-944525cc0f87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:53.952242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e37a77-3d83-4914-8950-944525cc0f87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:56:53.953862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e37a77-3d83-4914-8950-944525cc0f87 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:56:53.965595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1196/2419] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:56:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:56:55.100672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37619e64-c707-44cb-b498-6012f3bd852a service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:56:55.114489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37619e64-c707-44cb-b498-6012f3bd852a service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:56:55.120525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a58ca9-4d65-4d93-9d0b-0be47a64a98d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:56:55.121145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a58ca9-4d65-4d93-9d0b-0be47a64a98d service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:56:55.121425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a58ca9-4d65-4d93-9d0b-0be47a64a98d service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:56:55.121697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a58ca9-4d65-4d93-9d0b-0be47a64a98d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:56:55.123399 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d0a58ca9-4d65-4d93-9d0b-0be47a64a98d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:55.134201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a58ca9-4d65-4d93-9d0b-0be47a64a98d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:56:55.135893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a58ca9-4d65-4d93-9d0b-0be47a64a98d service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:56:55.144642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1224/2420] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:56:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:56:55.823098 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e37a77-3d83-4914-8950-944525cc0f87 service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:56:55.832341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e37a77-3d83-4914-8950-944525cc0f87 service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:56:55.835899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9b08-9701-41b3-8e85-7c4dcc155007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:56:55.836497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9b08-9701-41b3-8e85-7c4dcc155007 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:56:55.836777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9b08-9701-41b3-8e85-7c4dcc155007 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:56:55.837063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9b08-9701-41b3-8e85-7c4dcc155007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:56:55.838956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2e1b9b08-9701-41b3-8e85-7c4dcc155007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:55.905573 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b9b08-9701-41b3-8e85-7c4dcc155007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:56:55.906935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b9b08-9701-41b3-8e85-7c4dcc155007 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:56:55.917254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1197/2421] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:56:59.694584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a58ca9-4d65-4d93-9d0b-0be47a64a98d service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:56:59.700519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d0a58ca9-4d65-4d93-9d0b-0be47a64a98d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:59.772815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a58ca9-4d65-4d93-9d0b-0be47a64a98d service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:56:59.778777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040686bb-81ab-4c56-b91e-eccbd155c20b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:56:59.779387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040686bb-81ab-4c56-b91e-eccbd155c20b service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:56:59.779668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040686bb-81ab-4c56-b91e-eccbd155c20b service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:56:59.779940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040686bb-81ab-4c56-b91e-eccbd155c20b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:56:59.781678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-040686bb-81ab-4c56-b91e-eccbd155c20b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:59.794555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040686bb-81ab-4c56-b91e-eccbd155c20b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:56:59.796106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040686bb-81ab-4c56-b91e-eccbd155c20b service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:56:59.805867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1225/2422] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:56:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:56:59.849136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9b08-9701-41b3-8e85-7c4dcc155007 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:56:59.850874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2e1b9b08-9701-41b3-8e85-7c4dcc155007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:56:59.867354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9b08-9701-41b3-8e85-7c4dcc155007 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:56:59.868619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:56:59.869267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:56:59.869548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:56:59.869819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] PATH_INFO: `/v3/users/91e852d04f9f464ab8afbaeb9ff355e6` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:56:59.870927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e852d04f9f464ab8afbaeb9ff355e6)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:56:59.872851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:00.044329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e852d04f9f464ab8afbaeb9ff355e6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:00.044988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e852d04f9f464ab8afbaeb9ff355e6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:00.045643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e852d04f9f464ab8afbaeb9ff355e6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:00.058665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e852d04f9f464ab8afbaeb9ff355e6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:00.065469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d89268b8-ffdf-43ba-952a-93f58672314e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e852d04f9f464ab8afbaeb9ff355e6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:00.133283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1198/2423] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:56:59 2022] DELETE /identity/v3/users/91e852d04f9f464ab8afbaeb9ff355e6 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:57:00.390814 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040686bb-81ab-4c56-b91e-eccbd155c20b service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:00.403455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040686bb-81ab-4c56-b91e-eccbd155c20b service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:00.473295 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:00.473932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:00.474219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:00.474493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] PATH_INFO: `/v3/projects/2b95b79f22a341488f44fc5a840975c9` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:00.475631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b95b79f22a341488f44fc5a840975c9)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:00.477590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:00.583956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b95b79f22a341488f44fc5a840975c9', 'request_id': 'req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:00.590892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b95b79f22a341488f44fc5a840975c9', 'request_id': 'req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:00.663306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b95b79f22a341488f44fc5a840975c9', 'request_id': 'req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:00.667781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:57:00.675029 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] The token cache is being invalidate because project 2b95b79f22a341488f44fc5a840975c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:57:00.678338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:00.678338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-384c94ad-bf6f-4ed0-9c1c-7235f3e57d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:00.678338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1226/2424] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:57:00 2022] DELETE /identity/v3/projects/2b95b79f22a341488f44fc5a840975c9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:57:00.682452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee2a1-6ae1-41b3-9e38-1382bbee8750 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:00.682741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee2a1-6ae1-41b3-9e38-1382bbee8750 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:00.683017 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bee2a1-6ae1-41b3-9e38-1382bbee8750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:00.702715 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e0bee2a1-6ae1-41b3-9e38-1382bbee8750 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:00.703909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e0bee2a1-6ae1-41b3-9e38-1382bbee8750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:00.751292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:57:00.763834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1199/2425] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:00.771661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facf417-a7fd-4157-bb49-05a9dcb76181 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:00.772148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facf417-a7fd-4157-bb49-05a9dcb76181 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:00.773542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facf417-a7fd-4157-bb49-05a9dcb76181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:00.792824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1facf417-a7fd-4157-bb49-05a9dcb76181 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:00.796599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1facf417-a7fd-4157-bb49-05a9dcb76181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:00.802020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1227/2426] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:00.806191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bee2a1-6ae1-41b3-9e38-1382bbee8750 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:00.816245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bee2a1-6ae1-41b3-9e38-1382bbee8750 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:00.820946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5d86-f7f5-46a3-a6fe-8b86e81935f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:00.825245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5d86-f7f5-46a3-a6fe-8b86e81935f0 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:00.825245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5d86-f7f5-46a3-a6fe-8b86e81935f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:00.825245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5d86-f7f5-46a3-a6fe-8b86e81935f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:00.825245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5d86-f7f5-46a3-a6fe-8b86e81935f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:57:00.825245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5d86-f7f5-46a3-a6fe-8b86e81935f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:00.825245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5d86-f7f5-46a3-a6fe-8b86e81935f0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:00.836915 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1200/2427] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:57:00.841121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1facf417-a7fd-4157-bb49-05a9dcb76181 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:00.850302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1facf417-a7fd-4157-bb49-05a9dcb76181 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:00.855663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cddf4f-8ff2-4fb1-a994-76bd84e8a485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:00.856247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cddf4f-8ff2-4fb1-a994-76bd84e8a485 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:00.856521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cddf4f-8ff2-4fb1-a994-76bd84e8a485 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:00.856788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cddf4f-8ff2-4fb1-a994-76bd84e8a485 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:00.857508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cddf4f-8ff2-4fb1-a994-76bd84e8a485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:00.859383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cddf4f-8ff2-4fb1-a994-76bd84e8a485 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:00.917554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1228/2428] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:57:00 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:57:00.921840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5d86-f7f5-46a3-a6fe-8b86e81935f0 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:00.934877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5d86-f7f5-46a3-a6fe-8b86e81935f0 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:00.996522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277b4b5-59e3-42ca-8e53-259e3da140ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:00.997156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277b4b5-59e3-42ca-8e53-259e3da140ea admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:00.997440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277b4b5-59e3-42ca-8e53-259e3da140ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:00.997717 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277b4b5-59e3-42ca-8e53-259e3da140ea admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:00.998363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277b4b5-59e3-42ca-8e53-259e3da140ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:01.000251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277b4b5-59e3-42ca-8e53-259e3da140ea admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:01.087097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1201/2429] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:57:00 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:57:01.091742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cddf4f-8ff2-4fb1-a994-76bd84e8a485 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:01.102886 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cddf4f-8ff2-4fb1-a994-76bd84e8a485 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:01.108168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17949f4d-f6b0-4654-8bf8-e9958ecb9c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:01.108872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17949f4d-f6b0-4654-8bf8-e9958ecb9c38 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:01.109184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17949f4d-f6b0-4654-8bf8-e9958ecb9c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:01.109476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17949f4d-f6b0-4654-8bf8-e9958ecb9c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:01.110108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17949f4d-f6b0-4654-8bf8-e9958ecb9c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:01.111668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17949f4d-f6b0-4654-8bf8-e9958ecb9c38 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:01.121809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1229/2430] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:57:01.125834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277b4b5-59e3-42ca-8e53-259e3da140ea admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:01.135765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277b4b5-59e3-42ca-8e53-259e3da140ea admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:01.140586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017e705-a64b-4dcd-aa12-db93fd58bf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:01.141229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017e705-a64b-4dcd-aa12-db93fd58bf85 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:01.141513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017e705-a64b-4dcd-aa12-db93fd58bf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:01.141794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017e705-a64b-4dcd-aa12-db93fd58bf85 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:01.142428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017e705-a64b-4dcd-aa12-db93fd58bf85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:01.144033 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017e705-a64b-4dcd-aa12-db93fd58bf85 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:01.155521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1202/2431] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:57:01.159479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17949f4d-f6b0-4654-8bf8-e9958ecb9c38 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:01.170095 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17949f4d-f6b0-4654-8bf8-e9958ecb9c38 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:01.174650 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930798a3-f353-48bb-9f88-d34835558968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:01.175283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930798a3-f353-48bb-9f88-d34835558968 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:01.175565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930798a3-f353-48bb-9f88-d34835558968 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:01.175834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930798a3-f353-48bb-9f88-d34835558968 admin admin] PATH_INFO: `/v3/projects/e786d60ca59f4d23be47a4a8305123f4/users/1c04b545aa674beab518178a8936b788/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:01.190145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930798a3-f353-48bb-9f88-d34835558968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e786d60ca59f4d23be47a4a8305123f4, user_id=1c04b545aa674beab518178a8936b788, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:01.192753 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930798a3-f353-48bb-9f88-d34835558968 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:01.294189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1230/2432] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:57:01 2022] PUT /identity/v3/projects/e786d60ca59f4d23be47a4a8305123f4/users/1c04b545aa674beab518178a8936b788/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:57:01.299383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e5367-a480-4f2b-80e6-0eba0406dd2b None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:01.299670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e5367-a480-4f2b-80e6-0eba0406dd2b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:01.299948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e5367-a480-4f2b-80e6-0eba0406dd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:01.323219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-db2e5367-a480-4f2b-80e6-0eba0406dd2b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:01.324464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-db2e5367-a480-4f2b-80e6-0eba0406dd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:01.382613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1203/2433] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:01.393132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930798a3-f353-48bb-9f88-d34835558968 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:01.394815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-930798a3-f353-48bb-9f88-d34835558968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:01.452720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930798a3-f353-48bb-9f88-d34835558968 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:01.456437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acd892-582a-43fd-ac6d-43eea22692be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:01.459299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acd892-582a-43fd-ac6d-43eea22692be service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:01.459299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acd892-582a-43fd-ac6d-43eea22692be service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:01.459299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acd892-582a-43fd-ac6d-43eea22692be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:01.467818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acd892-582a-43fd-ac6d-43eea22692be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:01.469269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acd892-582a-43fd-ac6d-43eea22692be service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:01.478870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1231/2434] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:57:07.429962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bf6f2-7ddb-4a90-8f6c-754778bacac9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:07.430638 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bf6f2-7ddb-4a90-8f6c-754778bacac9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:07.431023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bf6f2-7ddb-4a90-8f6c-754778bacac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:07.469311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-243bf6f2-7ddb-4a90-8f6c-754778bacac9 None None] MFA Rules not processed for user `1c04b545aa674beab518178a8936b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:07.471508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-243bf6f2-7ddb-4a90-8f6c-754778bacac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:07.537507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:57:07.558919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1204/2435] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:57:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:07.574590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acd892-582a-43fd-ac6d-43eea22692be service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:07.576396 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f1acd892-582a-43fd-ac6d-43eea22692be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:07.681637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acd892-582a-43fd-ac6d-43eea22692be service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:07.688598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8d4d1-0631-47ea-85e5-762e1de49be9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:07.689330 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8d4d1-0631-47ea-85e5-762e1de49be9 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:07.689611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8d4d1-0631-47ea-85e5-762e1de49be9 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:07.689882 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8d4d1-0631-47ea-85e5-762e1de49be9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:07.699836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8d4d1-0631-47ea-85e5-762e1de49be9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:07.701693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8d4d1-0631-47ea-85e5-762e1de49be9 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:07.713888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1232/2436] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:57:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:57:08.116762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bf6f2-7ddb-4a90-8f6c-754778bacac9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:08.153576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bf6f2-7ddb-4a90-8f6c-754778bacac9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:08.160819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e3e08-3d28-4a4a-80c4-824f54d7ff7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:08.161069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e3e08-3d28-4a4a-80c4-824f54d7ff7e service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:08.161583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e3e08-3d28-4a4a-80c4-824f54d7ff7e service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:08.162251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e3e08-3d28-4a4a-80c4-824f54d7ff7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:08.164634 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5a2e3e08-3d28-4a4a-80c4-824f54d7ff7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:08.209226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e3e08-3d28-4a4a-80c4-824f54d7ff7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:08.212630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e3e08-3d28-4a4a-80c4-824f54d7ff7e service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:08.240673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1205/2437] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:57:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:57:10.688859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e91de-03c1-4bf9-9e73-cbfe429afcf2 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:10.689986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e91de-03c1-4bf9-9e73-cbfe429afcf2 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:10.689986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e91de-03c1-4bf9-9e73-cbfe429afcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:10.716547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-870e91de-03c1-4bf9-9e73-cbfe429afcf2 None None] MFA Rules not processed for user `1c04b545aa674beab518178a8936b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:10.717366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-870e91de-03c1-4bf9-9e73-cbfe429afcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:10.724597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1233/2438] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:57:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:11.133515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e3e08-3d28-4a4a-80c4-824f54d7ff7e service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:11.139202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5a2e3e08-3d28-4a4a-80c4-824f54d7ff7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:11.154790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e3e08-3d28-4a4a-80c4-824f54d7ff7e service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:11.160801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833986ad-0951-4043-a6f9-737b59ae49a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:11.161527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833986ad-0951-4043-a6f9-737b59ae49a5 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:11.162485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833986ad-0951-4043-a6f9-737b59ae49a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:11.162810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833986ad-0951-4043-a6f9-737b59ae49a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:11.168334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-833986ad-0951-4043-a6f9-737b59ae49a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:11.179409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833986ad-0951-4043-a6f9-737b59ae49a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:11.181085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833986ad-0951-4043-a6f9-737b59ae49a5 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:11.191867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1206/2439] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:57:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:57:11.573668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e91de-03c1-4bf9-9e73-cbfe429afcf2 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:11.575065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-870e91de-03c1-4bf9-9e73-cbfe429afcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:11.589616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:57:11.666152 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e91de-03c1-4bf9-9e73-cbfe429afcf2 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:11.673010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7e55f-b953-4fde-8010-18aab3102f23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:11.673647 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7e55f-b953-4fde-8010-18aab3102f23 service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:11.673930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7e55f-b953-4fde-8010-18aab3102f23 service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:11.674202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7e55f-b953-4fde-8010-18aab3102f23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:11.675928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a9f7e55f-b953-4fde-8010-18aab3102f23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:11.687848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7e55f-b953-4fde-8010-18aab3102f23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:11.689447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7e55f-b953-4fde-8010-18aab3102f23 service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:11.699903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1234/2440] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:57:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:57:17.547027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb5a04-b500-4dac-9ba0-b58eec9b97f1 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:17.548739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb5a04-b500-4dac-9ba0-b58eec9b97f1 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:17.548739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb5a04-b500-4dac-9ba0-b58eec9b97f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:17.574098 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-59eb5a04-b500-4dac-9ba0-b58eec9b97f1 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:17.575002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-59eb5a04-b500-4dac-9ba0-b58eec9b97f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:17.657525 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1207/2441] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 18:57:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:57:17.789217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7e55f-b953-4fde-8010-18aab3102f23 service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:17.791148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a9f7e55f-b953-4fde-8010-18aab3102f23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:17.869956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7e55f-b953-4fde-8010-18aab3102f23 service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:17.876236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f74602-d478-4eab-9b6c-fda5857c9fe1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:17.876860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f74602-d478-4eab-9b6c-fda5857c9fe1 service swift] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:17.877193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f74602-d478-4eab-9b6c-fda5857c9fe1 service swift] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:17.877459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f74602-d478-4eab-9b6c-fda5857c9fe1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:17.886346 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f74602-d478-4eab-9b6c-fda5857c9fe1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:17.888084 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f74602-d478-4eab-9b6c-fda5857c9fe1 service swift] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:17.896613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1235/2442] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:57:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 18:57:18.140719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cbefb-8118-44cd-9bf4-c8bcbc175d38 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:18.141001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cbefb-8118-44cd-9bf4-c8bcbc175d38 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:18.141306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cbefb-8118-44cd-9bf4-c8bcbc175d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:18.161938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-2e3cbefb-8118-44cd-9bf4-c8bcbc175d38 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:18.163147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2e3cbefb-8118-44cd-9bf4-c8bcbc175d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:18.171150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1208/2443] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:57:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:18.184110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f74602-d478-4eab-9b6c-fda5857c9fe1 service swift] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:18.194767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f74602-d478-4eab-9b6c-fda5857c9fe1 service swift] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:18.200532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33f6a-a25f-45cc-9076-ce701f27b0ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:18.201146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33f6a-a25f-45cc-9076-ce701f27b0ed service swift] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:18.205070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33f6a-a25f-45cc-9076-ce701f27b0ed service swift] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:18.205070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33f6a-a25f-45cc-9076-ce701f27b0ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:18.209994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33f6a-a25f-45cc-9076-ce701f27b0ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:18.212086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33f6a-a25f-45cc-9076-ce701f27b0ed service swift] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:18.221247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1236/2444] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:57:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 18:57:19.243849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cbefb-8118-44cd-9bf4-c8bcbc175d38 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:19.246125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2e3cbefb-8118-44cd-9bf4-c8bcbc175d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:19.269560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cbefb-8118-44cd-9bf4-c8bcbc175d38 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:19.276373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb848f-6c98-4333-a7d5-a6944f3778ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:19.277307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb848f-6c98-4333-a7d5-a6944f3778ec service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:19.277825 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb848f-6c98-4333-a7d5-a6944f3778ec service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:19.278291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb848f-6c98-4333-a7d5-a6944f3778ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:19.280909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3dbb848f-6c98-4333-a7d5-a6944f3778ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:19.295798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb848f-6c98-4333-a7d5-a6944f3778ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:19.297905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb848f-6c98-4333-a7d5-a6944f3778ec service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:19.314345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1209/2445] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:57:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:57:28.219107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=65 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:57:28.307024 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33f6a-a25f-45cc-9076-ce701f27b0ed service swift] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:28.309219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7bd33f6a-a25f-45cc-9076-ce701f27b0ed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:28.340331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33f6a-a25f-45cc-9076-ce701f27b0ed service swift] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:28.347373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a2969-57d1-46cd-9dc7-f6477c5cda08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:28.348266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a2969-57d1-46cd-9dc7-f6477c5cda08 service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:28.353516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a2969-57d1-46cd-9dc7-f6477c5cda08 service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:28.353516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a2969-57d1-46cd-9dc7-f6477c5cda08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:28.353516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3b0a2969-57d1-46cd-9dc7-f6477c5cda08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:28.372180 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a2969-57d1-46cd-9dc7-f6477c5cda08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:28.374032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a2969-57d1-46cd-9dc7-f6477c5cda08 service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:28.388676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1237/2446] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:57:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:57:28.443181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb848f-6c98-4333-a7d5-a6944f3778ec service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:28.445516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3dbb848f-6c98-4333-a7d5-a6944f3778ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:28.471400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb848f-6c98-4333-a7d5-a6944f3778ec service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:28.479858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:28.480655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:28.481196 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:28.481706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] PATH_INFO: `/v3/users/f7555f03f8e34998ab1890279bf9b5c8` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:28.483047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7555f03f8e34998ab1890279bf9b5c8)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:28.485087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:28.662381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7555f03f8e34998ab1890279bf9b5c8'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:28.663429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7555f03f8e34998ab1890279bf9b5c8'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:28.664354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7555f03f8e34998ab1890279bf9b5c8'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:28.680581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7555f03f8e34998ab1890279bf9b5c8'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:28.690738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7555f03f8e34998ab1890279bf9b5c8'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:28.771857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1210/2447] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:57:28 2022] DELETE /identity/v3/users/f7555f03f8e34998ab1890279bf9b5c8 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:57:29.084592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a2969-57d1-46cd-9dc7-f6477c5cda08 service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:29.122774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a2969-57d1-46cd-9dc7-f6477c5cda08 service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:29.200790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:29.201590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:29.201973 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:29.202306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] PATH_INFO: `/v3/projects/a9c50a271b504ced86c125625e44c19e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:29.203745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9c50a271b504ced86c125625e44c19e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:29.205788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:29.751142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9c50a271b504ced86c125625e44c19e', 'request_id': 'req-e3910a38-2426-4116-9b14-c7d37c3a5b2f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:29.757746 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9c50a271b504ced86c125625e44c19e', 'request_id': 'req-e3910a38-2426-4116-9b14-c7d37c3a5b2f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:29.841511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9c50a271b504ced86c125625e44c19e', 'request_id': 'req-e3910a38-2426-4116-9b14-c7d37c3a5b2f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:29.855151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] The token cache is being invalidate because project a9c50a271b504ced86c125625e44c19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:57:29.855421 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:29.856313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:29.857856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1238/2448] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:57:29 2022] DELETE /identity/v3/projects/a9c50a271b504ced86c125625e44c19e => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:57:29.865026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:29.866923 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:29.880839 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f02aa-5763-482c-917a-a99ed2f90bd9 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:29.900703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:57:29.944948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:29.945620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:29.945897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:29.946348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] PATH_INFO: `/v3/users/72f7e33f9a06403ab2c3c5fb553f6978` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:29.947700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f7e33f9a06403ab2c3c5fb553f6978)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:29.950370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:30.141826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f7e33f9a06403ab2c3c5fb553f6978'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:30.142684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f7e33f9a06403ab2c3c5fb553f6978'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:30.143369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f7e33f9a06403ab2c3c5fb553f6978'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:30.159548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f7e33f9a06403ab2c3c5fb553f6978'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:30.177739 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ad6243c0-7bca-4917-b866-70560bc2e35a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f7e33f9a06403ab2c3c5fb553f6978'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:30.303758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1211/2449] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:57:29 2022] DELETE /identity/v3/users/72f7e33f9a06403ab2c3c5fb553f6978 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:57:30.614570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:30.618812 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:30.647027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3910a38-2426-4116-9b14-c7d37c3a5b2f admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:30.744804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:30.745549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:30.745832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:30.746340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] PATH_INFO: `/v3/projects/aac7e6ed26fa439bb2d44d462b70c6d9` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:30.747512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac7e6ed26fa439bb2d44d462b70c6d9)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:30.749459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:30.773389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:57:30.913507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac7e6ed26fa439bb2d44d462b70c6d9', 'request_id': 'req-e585ef16-3da9-45b3-9743-d627d062fa39'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:30.921382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac7e6ed26fa439bb2d44d462b70c6d9', 'request_id': 'req-e585ef16-3da9-45b3-9743-d627d062fa39'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:31.005733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac7e6ed26fa439bb2d44d462b70c6d9', 'request_id': 'req-e585ef16-3da9-45b3-9743-d627d062fa39'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:31.018234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] The token cache is being invalidate because project aac7e6ed26fa439bb2d44d462b70c6d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:57:31.018602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:31.019440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e585ef16-3da9-45b3-9743-d627d062fa39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:57:31.020983 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1239/2450] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:57:30 2022] DELETE /identity/v3/projects/aac7e6ed26fa439bb2d44d462b70c6d9 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:57:37.974784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720becc-9f7c-4532-9f11-e82ca9d3e65d None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:37.975207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720becc-9f7c-4532-9f11-e82ca9d3e65d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:37.975320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720becc-9f7c-4532-9f11-e82ca9d3e65d None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:37.976637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1212/2451] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:57:37.982732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2b8c1-4478-42d3-9c10-a922a4456fb0 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:37.983334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2b8c1-4478-42d3-9c10-a922a4456fb0 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:37.983432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2b8c1-4478-42d3-9c10-a922a4456fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:38.004533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f8c2b8c1-4478-42d3-9c10-a922a4456fb0 None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:38.005818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f8c2b8c1-4478-42d3-9c10-a922a4456fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:38.073429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1240/2452] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:57:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:38.085958 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720becc-9f7c-4532-9f11-e82ca9d3e65d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:38.087623 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5720becc-9f7c-4532-9f11-e82ca9d3e65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:38.145552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720becc-9f7c-4532-9f11-e82ca9d3e65d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:38.149106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d52da4-d987-4691-909a-e60e5620a8ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8d10585480f41608bc6935e282f7e27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:38.149696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d52da4-d987-4691-909a-e60e5620a8ef service glance] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:38.149974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d52da4-d987-4691-909a-e60e5620a8ef service glance] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:38.150239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d52da4-d987-4691-909a-e60e5620a8ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:38.160700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d52da4-d987-4691-909a-e60e5620a8ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:38.162171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d52da4-d987-4691-909a-e60e5620a8ef service glance] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:38.172745 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1213/2453] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:57:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:57:38.363196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428320d-e19e-4e1a-b438-c097f64c9c12 None None] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:38.363485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428320d-e19e-4e1a-b438-c097f64c9c12 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:38.363763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428320d-e19e-4e1a-b438-c097f64c9c12 None None] PATH_INFO: `/` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:38.365178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1241/2454] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:57:38.370243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edaad7-b252-47af-bf0f-f8e03125111e None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:38.370555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edaad7-b252-47af-bf0f-f8e03125111e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:38.370832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edaad7-b252-47af-bf0f-f8e03125111e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:38.391878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-76edaad7-b252-47af-bf0f-f8e03125111e None None] MFA Rules not processed for user `bcf30ef695154d3a97d7e589aca21233`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:38.393110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-76edaad7-b252-47af-bf0f-f8e03125111e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:38.431077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:57:38.444096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1214/2455] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:57:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:38.456762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428320d-e19e-4e1a-b438-c097f64c9c12 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:38.458509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0428320d-e19e-4e1a-b438-c097f64c9c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:38.471793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428320d-e19e-4e1a-b438-c097f64c9c12 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:38.475314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbf0d8-48d8-49c6-a2e2-934df626dc46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:38.475908 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf0d8-48d8-49c6-a2e2-934df626dc46 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:38.476191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf0d8-48d8-49c6-a2e2-934df626dc46 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:38.476465 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf0d8-48d8-49c6-a2e2-934df626dc46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:38.483540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbf0d8-48d8-49c6-a2e2-934df626dc46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:38.484943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbf0d8-48d8-49c6-a2e2-934df626dc46 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:38.493011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1242/2456] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:57:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:57:38.579750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b769c-73b2-4f14-b43e-a891d4c55968 None None] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:38.580039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b769c-73b2-4f14-b43e-a891d4c55968 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:38.580321 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b769c-73b2-4f14-b43e-a891d4c55968 None None] PATH_INFO: `/` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:38.581740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1215/2457] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 8 18:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 18:57:38.586616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c19a5-29cd-4ca4-aa53-641272ca710b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:38.586902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c19a5-29cd-4ca4-aa53-641272ca710b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:38.587179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c19a5-29cd-4ca4-aa53-641272ca710b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:38.606776 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-104c19a5-29cd-4ca4-aa53-641272ca710b None None] MFA Rules not processed for user `1b29c5745a8e42ff8232e5ca22b4a4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:38.608000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-104c19a5-29cd-4ca4-aa53-641272ca710b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:38.617532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1243/2458] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 8 18:57:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:38.628708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b769c-73b2-4f14-b43e-a891d4c55968 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:38.640352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b769c-73b2-4f14-b43e-a891d4c55968 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:38.644031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b808a-7efe-481a-a307-e3d29f9cd7d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:38.644627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b808a-7efe-481a-a307-e3d29f9cd7d7 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:38.644909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b808a-7efe-481a-a307-e3d29f9cd7d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:38.645224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b808a-7efe-481a-a307-e3d29f9cd7d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:38.655566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b808a-7efe-481a-a307-e3d29f9cd7d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:38.657000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b808a-7efe-481a-a307-e3d29f9cd7d7 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:38.667717 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1216/2459] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:57:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:57:41.252689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdac92-8b64-4dfb-b0e9-a3e645848c94 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:41.253214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdac92-8b64-4dfb-b0e9-a3e645848c94 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:41.253310 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdac92-8b64-4dfb-b0e9-a3e645848c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:41.272870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-cdfdac92-8b64-4dfb-b0e9-a3e645848c94 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:41.274168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-cdfdac92-8b64-4dfb-b0e9-a3e645848c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:41.329554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1244/2460] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 18:57:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:41.344505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b808a-7efe-481a-a307-e3d29f9cd7d7 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:41.345998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f78b808a-7efe-481a-a307-e3d29f9cd7d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:41.414813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b808a-7efe-481a-a307-e3d29f9cd7d7 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:41.419801 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce7a9c-04e4-4b03-a821-5ed73a28307b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:41.420565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce7a9c-04e4-4b03-a821-5ed73a28307b service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:41.420887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce7a9c-04e4-4b03-a821-5ed73a28307b service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:41.421223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce7a9c-04e4-4b03-a821-5ed73a28307b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:41.429184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce7a9c-04e4-4b03-a821-5ed73a28307b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:41.430768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce7a9c-04e4-4b03-a821-5ed73a28307b service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:41.439516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1217/2461] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:57:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 18:57:51.321927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:57:51.435557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=42 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:57:58.854515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a5f60-8fe7-438d-b76f-2f29eaa37ff3 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:58.855032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a5f60-8fe7-438d-b76f-2f29eaa37ff3 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:58.855032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a5f60-8fe7-438d-b76f-2f29eaa37ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:58.874741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a52a5f60-8fe7-438d-b76f-2f29eaa37ff3 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:57:58.875944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a52a5f60-8fe7-438d-b76f-2f29eaa37ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:57:58.883251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1245/2462] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 18:57:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:57:58.897377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce7a9c-04e4-4b03-a821-5ed73a28307b service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:57:58.908186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce7a9c-04e4-4b03-a821-5ed73a28307b service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:57:58.913974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217ee54-ec28-4f76-93ce-c130a58d8a97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:57:58.914586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217ee54-ec28-4f76-93ce-c130a58d8a97 service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:57:58.914864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217ee54-ec28-4f76-93ce-c130a58d8a97 service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:57:58.915212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217ee54-ec28-4f76-93ce-c130a58d8a97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:57:58.924034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217ee54-ec28-4f76-93ce-c130a58d8a97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:57:58.925710 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217ee54-ec28-4f76-93ce-c130a58d8a97 service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:57:58.934509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1218/2463] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 18:57:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 18:58:03.100512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1edbbf-cb26-4403-809d-a2777225db22 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:03.101034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1edbbf-cb26-4403-809d-a2777225db22 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:03.101169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1edbbf-cb26-4403-809d-a2777225db22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:03.120316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2e1edbbf-cb26-4403-809d-a2777225db22 None None] MFA Rules not processed for user `8da410d2ea0740dc8eeea87244fc1408`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:58:03.122254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2e1edbbf-cb26-4403-809d-a2777225db22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:03.177424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1246/2464] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:58:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:58:05.995506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217ee54-ec28-4f76-93ce-c130a58d8a97 service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:06.005683 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217ee54-ec28-4f76-93ce-c130a58d8a97 service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:06.009709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a09eeb-4e2c-45dc-a4b4-21b7c9135607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:06.010315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a09eeb-4e2c-45dc-a4b4-21b7c9135607 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:06.010695 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a09eeb-4e2c-45dc-a4b4-21b7c9135607 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:06.010887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a09eeb-4e2c-45dc-a4b4-21b7c9135607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:06.012782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-56a09eeb-4e2c-45dc-a4b4-21b7c9135607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:06.024219 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a09eeb-4e2c-45dc-a4b4-21b7c9135607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:06.025758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a09eeb-4e2c-45dc-a4b4-21b7c9135607 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:06.037225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1219/2465] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:58:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:58:06.041617 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1edbbf-cb26-4403-809d-a2777225db22 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:06.043178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2e1edbbf-cb26-4403-809d-a2777225db22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:06.060453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1edbbf-cb26-4403-809d-a2777225db22 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:06.066263 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec96b9-444b-4a10-9300-56d5efe4dbd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:06.066884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec96b9-444b-4a10-9300-56d5efe4dbd1 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:06.067161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec96b9-444b-4a10-9300-56d5efe4dbd1 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:06.067432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec96b9-444b-4a10-9300-56d5efe4dbd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:06.075669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec96b9-444b-4a10-9300-56d5efe4dbd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:06.077266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec96b9-444b-4a10-9300-56d5efe4dbd1 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:06.086540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1247/2466] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:58:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:58:11.616630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a09eeb-4e2c-45dc-a4b4-21b7c9135607 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:11.618388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-56a09eeb-4e2c-45dc-a4b4-21b7c9135607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:11.629589 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a09eeb-4e2c-45dc-a4b4-21b7c9135607 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:11.633144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70e843-5f1c-45d4-b0fa-64aba6f7a684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:11.633735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70e843-5f1c-45d4-b0fa-64aba6f7a684 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:11.634012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70e843-5f1c-45d4-b0fa-64aba6f7a684 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:11.634309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70e843-5f1c-45d4-b0fa-64aba6f7a684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:11.635891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bb70e843-5f1c-45d4-b0fa-64aba6f7a684 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:11.696919 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70e843-5f1c-45d4-b0fa-64aba6f7a684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:11.698384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70e843-5f1c-45d4-b0fa-64aba6f7a684 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:11.708330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1220/2467] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:58:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:58:16.082873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=34 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:18.577077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec96b9-444b-4a10-9300-56d5efe4dbd1 service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:18.589367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec96b9-444b-4a10-9300-56d5efe4dbd1 service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:18.596775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c630fa-3270-4066-a62b-f3fe81ca0f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:18.597966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c630fa-3270-4066-a62b-f3fe81ca0f48 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:18.599442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c630fa-3270-4066-a62b-f3fe81ca0f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:18.599442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c630fa-3270-4066-a62b-f3fe81ca0f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:18.600758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-82c630fa-3270-4066-a62b-f3fe81ca0f48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:18.684340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c630fa-3270-4066-a62b-f3fe81ca0f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:18.685975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c630fa-3270-4066-a62b-f3fe81ca0f48 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:18.697240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1248/2468] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:58:21.703147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=40 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:26.533933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70e843-5f1c-45d4-b0fa-64aba6f7a684 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:26.535391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bb70e843-5f1c-45d4-b0fa-64aba6f7a684 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:26.550712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70e843-5f1c-45d4-b0fa-64aba6f7a684 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:26.555284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:26.555910 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:26.556188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:26.556462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] PATH_INFO: `/v3/users/db0a62db116947df80b484dde76eab60` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:26.557614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db0a62db116947df80b484dde76eab60)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:26.559592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:26.763744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db0a62db116947df80b484dde76eab60'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:26.764407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db0a62db116947df80b484dde76eab60'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:26.765023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db0a62db116947df80b484dde76eab60'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:26.777392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db0a62db116947df80b484dde76eab60'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:26.788342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db0a62db116947df80b484dde76eab60'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:26.869382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1221/2469] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:58:26 2022] DELETE /identity/v3/users/db0a62db116947df80b484dde76eab60 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:58:27.132562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c630fa-3270-4066-a62b-f3fe81ca0f48 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:27.147437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c630fa-3270-4066-a62b-f3fe81ca0f48 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:27.204191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:27.205872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:27.206158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:27.206437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] PATH_INFO: `/v3/projects/0833a02fb67046aeb7f70a317e9a2e73` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:27.207556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0833a02fb67046aeb7f70a317e9a2e73)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:27.209568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:27.305567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0833a02fb67046aeb7f70a317e9a2e73', 'request_id': 'req-e04e9324-57d1-4f06-ad88-ecb18d35e958'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:27.312211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0833a02fb67046aeb7f70a317e9a2e73', 'request_id': 'req-e04e9324-57d1-4f06-ad88-ecb18d35e958'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:27.378769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0833a02fb67046aeb7f70a317e9a2e73', 'request_id': 'req-e04e9324-57d1-4f06-ad88-ecb18d35e958'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:27.389899 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] The token cache is being invalidate because project 0833a02fb67046aeb7f70a317e9a2e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:58:27.390083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:27.390776 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:27.392921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1249/2470] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:58:27 2022] DELETE /identity/v3/projects/0833a02fb67046aeb7f70a317e9a2e73 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:27.397061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:27.398473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:27.412551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0190b9a-701a-4c11-a4d1-5739c4ff7b68 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:27.470731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:27.471372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:27.471664 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:27.471923 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] PATH_INFO: `/v3/users/5fb981e9468749f08821c3c45863cabc` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:27.473036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb981e9468749f08821c3c45863cabc)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:27.475100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:27.655808 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb981e9468749f08821c3c45863cabc'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:27.656457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb981e9468749f08821c3c45863cabc'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:27.656985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb981e9468749f08821c3c45863cabc'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:27.670684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb981e9468749f08821c3c45863cabc'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:27.677748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb981e9468749f08821c3c45863cabc'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:27.679386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:27.744135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1222/2471] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:58:27 2022] DELETE /identity/v3/users/5fb981e9468749f08821c3c45863cabc => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:58:27.991619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:27.993352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:28.006838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e9324-57d1-4f06-ad88-ecb18d35e958 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:28.028014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:28.066413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:28.067074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:28.067335 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:28.067609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] PATH_INFO: `/v3/projects/993dd93a784c403085f8ced9b07e9a91` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:28.068784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993dd93a784c403085f8ced9b07e9a91)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:28.070715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:28.168502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993dd93a784c403085f8ced9b07e9a91', 'request_id': 'req-542ef2fa-63e6-41dc-881d-b106f72c8f11'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:28.175211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993dd93a784c403085f8ced9b07e9a91', 'request_id': 'req-542ef2fa-63e6-41dc-881d-b106f72c8f11'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:28.227132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993dd93a784c403085f8ced9b07e9a91', 'request_id': 'req-542ef2fa-63e6-41dc-881d-b106f72c8f11'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:28.237632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] The token cache is being invalidate because project 993dd93a784c403085f8ced9b07e9a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:58:28.237857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:28.238476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:28.239946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1250/2472] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:58:27 2022] DELETE /identity/v3/projects/993dd93a784c403085f8ced9b07e9a91 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:37.801556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:38.234548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:42.845632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:42.847140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:42.906460 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7d866-8521-4605-91f6-bc025687af04 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:42.955964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93e88c-f05f-4e0f-9336-27ffaedf4855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:42.956570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93e88c-f05f-4e0f-9336-27ffaedf4855 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:42.956853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93e88c-f05f-4e0f-9336-27ffaedf4855 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:42.957735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93e88c-f05f-4e0f-9336-27ffaedf4855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:42.959493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8a93e88c-f05f-4e0f-9336-27ffaedf4855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:43.024491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93e88c-f05f-4e0f-9336-27ffaedf4855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:43.025938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a93e88c-f05f-4e0f-9336-27ffaedf4855 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:43.035993 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1223/2473] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:58:46.658520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:46.660567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:46.680109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ef2fa-63e6-41dc-881d-b106f72c8f11 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:46.682901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:46.683547 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:46.683828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:46.684106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] PATH_INFO: `/v3/users/2b57c46bc37940718735201095453f16` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:46.685267 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b57c46bc37940718735201095453f16)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:46.687201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:46.932480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b57c46bc37940718735201095453f16'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:46.933093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b57c46bc37940718735201095453f16'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:46.933711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b57c46bc37940718735201095453f16'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:46.940765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b57c46bc37940718735201095453f16'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:46.953231 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-c9144ddb-4ab8-4066-a1f6-32b796be843e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b57c46bc37940718735201095453f16'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:47.056102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1251/2474] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:58:46 2022] DELETE /identity/v3/users/2b57c46bc37940718735201095453f16 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:47.312138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93e88c-f05f-4e0f-9336-27ffaedf4855 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:47.320803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a93e88c-f05f-4e0f-9336-27ffaedf4855 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:47.380652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:47.381358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:47.383789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:47.383789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] PATH_INFO: `/v3/projects/9f381357ec29474f97a5b353daec21d5` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:47.383789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f381357ec29474f97a5b353daec21d5)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:47.385058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:47.393211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:47.497971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f381357ec29474f97a5b353daec21d5', 'request_id': 'req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:47.504867 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f381357ec29474f97a5b353daec21d5', 'request_id': 'req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:47.510562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f381357ec29474f97a5b353daec21d5', 'request_id': 'req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:47.640826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] The token cache is being invalidate because project 9f381357ec29474f97a5b353daec21d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:58:47.641154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:47.645785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-64ea7dcd-0394-4c31-a4b9-ddaa36904b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:47.645785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1224/2475] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:58:47 2022] DELETE /identity/v3/projects/9f381357ec29474f97a5b353daec21d5 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:47.649116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbe1f2-d771-475b-8e9b-0ad42e8a3a1f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:47.649413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbe1f2-d771-475b-8e9b-0ad42e8a3a1f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:47.649699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbe1f2-d771-475b-8e9b-0ad42e8a3a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:47.670485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-44dbe1f2-d771-475b-8e9b-0ad42e8a3a1f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:58:47.671718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-44dbe1f2-d771-475b-8e9b-0ad42e8a3a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:47.733006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1252/2476] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:58:47.741493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f930ad-45c5-4d4a-ab93-0204ad259bcb None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:47.741850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f930ad-45c5-4d4a-ab93-0204ad259bcb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:47.742190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f930ad-45c5-4d4a-ab93-0204ad259bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:47.763648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-62f930ad-45c5-4d4a-ab93-0204ad259bcb None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:58:47.765021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-62f930ad-45c5-4d4a-ab93-0204ad259bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:47.773340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1225/2477] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:58:47.777658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbe1f2-d771-475b-8e9b-0ad42e8a3a1f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:47.788041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbe1f2-d771-475b-8e9b-0ad42e8a3a1f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:47.793435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1739c-ba06-453d-9fe0-be68920e801d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:47.794042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1739c-ba06-453d-9fe0-be68920e801d admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:47.794319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1739c-ba06-453d-9fe0-be68920e801d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:47.794590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1739c-ba06-453d-9fe0-be68920e801d admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:47.795469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1739c-ba06-453d-9fe0-be68920e801d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:58:47.795604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1739c-ba06-453d-9fe0-be68920e801d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:47.797126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1739c-ba06-453d-9fe0-be68920e801d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:47.810074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1253/2478] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:58:47.814310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f930ad-45c5-4d4a-ab93-0204ad259bcb None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:47.824904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f930ad-45c5-4d4a-ab93-0204ad259bcb None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:47.829841 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30e464-f151-4b69-8785-2504cb1402ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:47.830469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e464-f151-4b69-8785-2504cb1402ef admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:47.830751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e464-f151-4b69-8785-2504cb1402ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:47.831026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30e464-f151-4b69-8785-2504cb1402ef admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:47.831704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30e464-f151-4b69-8785-2504cb1402ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:47.834703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30e464-f151-4b69-8785-2504cb1402ef admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:47.955210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1226/2479] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:58:47 2022] POST /identity/v3/projects => generated 403 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:58:47.960281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1739c-ba06-453d-9fe0-be68920e801d admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:47.971376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1739c-ba06-453d-9fe0-be68920e801d admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:48.029031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25ffff-f238-4648-9de2-9520083810c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:48.029657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25ffff-f238-4648-9de2-9520083810c2 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:48.029936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25ffff-f238-4648-9de2-9520083810c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:48.030205 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25ffff-f238-4648-9de2-9520083810c2 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:48.030858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25ffff-f238-4648-9de2-9520083810c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:48.032804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25ffff-f238-4648-9de2-9520083810c2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:48.040820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:48.138034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1254/2480] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:58:47 2022] POST /identity/v3/users => generated 302 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:58:48.142001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30e464-f151-4b69-8785-2504cb1402ef admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:48.153744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30e464-f151-4b69-8785-2504cb1402ef admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:48.171518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b624db-f0ff-4c37-a879-c5d8ea22e935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:48.172131 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b624db-f0ff-4c37-a879-c5d8ea22e935 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:48.172416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b624db-f0ff-4c37-a879-c5d8ea22e935 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:48.172705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b624db-f0ff-4c37-a879-c5d8ea22e935 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:48.173374 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b624db-f0ff-4c37-a879-c5d8ea22e935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:48.174980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b624db-f0ff-4c37-a879-c5d8ea22e935 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:48.187040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1227/2481] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:58:48.191077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25ffff-f238-4648-9de2-9520083810c2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:48.201963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25ffff-f238-4648-9de2-9520083810c2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:48.206679 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a93c86-6c7e-4650-a3a3-f5df296f51c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:48.207876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a93c86-6c7e-4650-a3a3-f5df296f51c7 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:48.208158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a93c86-6c7e-4650-a3a3-f5df296f51c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:48.208483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a93c86-6c7e-4650-a3a3-f5df296f51c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:48.209162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a93c86-6c7e-4650-a3a3-f5df296f51c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:48.210744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a93c86-6c7e-4650-a3a3-f5df296f51c7 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:48.222234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1255/2482] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:58:48.226307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b624db-f0ff-4c37-a879-c5d8ea22e935 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:48.236749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b624db-f0ff-4c37-a879-c5d8ea22e935 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:48.241718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f0714-152a-49a0-9bdd-c850a4c8c245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:48.242378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f0714-152a-49a0-9bdd-c850a4c8c245 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:48.242924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f0714-152a-49a0-9bdd-c850a4c8c245 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:48.243199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f0714-152a-49a0-9bdd-c850a4c8c245 admin admin] PATH_INFO: `/v3/projects/ee44fc6001a5478fb48c2ed545ec81d2/users/4626b77e9aaf4335b030b4c92c64afad/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:48.259432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f0714-152a-49a0-9bdd-c850a4c8c245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee44fc6001a5478fb48c2ed545ec81d2, user_id=4626b77e9aaf4335b030b4c92c64afad, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:48.262214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f0714-152a-49a0-9bdd-c850a4c8c245 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:48.348356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1228/2483] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:58:48 2022] PUT /identity/v3/projects/ee44fc6001a5478fb48c2ed545ec81d2/users/4626b77e9aaf4335b030b4c92c64afad/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:48.359928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05983ad9-0fea-408a-b57e-d33e4983176b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:48.360215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05983ad9-0fea-408a-b57e-d33e4983176b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:48.360494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05983ad9-0fea-408a-b57e-d33e4983176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:48.389896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-05983ad9-0fea-408a-b57e-d33e4983176b None None] MFA Rules not processed for user `4626b77e9aaf4335b030b4c92c64afad`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:58:48.391156 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-05983ad9-0fea-408a-b57e-d33e4983176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:48.459057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1256/2484] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:58:48 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:58:48.463703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f0714-152a-49a0-9bdd-c850a4c8c245 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:48.525695 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f0714-152a-49a0-9bdd-c850a4c8c245 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:48.530590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe3997-0fa9-4a3b-93e8-5e1c0020806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:48.531198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe3997-0fa9-4a3b-93e8-5e1c0020806a admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:48.531480 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe3997-0fa9-4a3b-93e8-5e1c0020806a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:48.531753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe3997-0fa9-4a3b-93e8-5e1c0020806a admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:48.532379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe3997-0fa9-4a3b-93e8-5e1c0020806a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:48.534359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe3997-0fa9-4a3b-93e8-5e1c0020806a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:48.621156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1229/2485] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:58:48 2022] POST /identity/v3/projects => generated 403 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:58:48.622538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05983ad9-0fea-408a-b57e-d33e4983176b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:48.653754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05983ad9-0fea-408a-b57e-d33e4983176b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:48.717686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9012b7-0615-4708-8104-3b138e0dc774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:48.718358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9012b7-0615-4708-8104-3b138e0dc774 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:48.718619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9012b7-0615-4708-8104-3b138e0dc774 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:48.718892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9012b7-0615-4708-8104-3b138e0dc774 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:48.719517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9012b7-0615-4708-8104-3b138e0dc774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:48.721460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9012b7-0615-4708-8104-3b138e0dc774 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:48.807275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1257/2486] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:58:48 2022] POST /identity/v3/users => generated 308 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:58:48.811714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe3997-0fa9-4a3b-93e8-5e1c0020806a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:48.823385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe3997-0fa9-4a3b-93e8-5e1c0020806a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:48.828320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68480618-606d-45b4-9ebf-49388a476f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:48.828991 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68480618-606d-45b4-9ebf-49388a476f9f admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:48.829373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68480618-606d-45b4-9ebf-49388a476f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:48.829714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68480618-606d-45b4-9ebf-49388a476f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:48.830437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68480618-606d-45b4-9ebf-49388a476f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:48.832912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68480618-606d-45b4-9ebf-49388a476f9f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:48.847418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1230/2487] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:58:48.848823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9012b7-0615-4708-8104-3b138e0dc774 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:48.859279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9012b7-0615-4708-8104-3b138e0dc774 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:48.865667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f969ab-927c-4ddf-92c7-e4bb3753e595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:48.866343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f969ab-927c-4ddf-92c7-e4bb3753e595 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:48.866644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f969ab-927c-4ddf-92c7-e4bb3753e595 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:48.866908 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f969ab-927c-4ddf-92c7-e4bb3753e595 admin admin] PATH_INFO: `/v3/projects/a3324661f420412ba23f9a9bd16fc842/users/cab064e21e0745f79ea217b3bcf1ffc1/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:48.883144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f969ab-927c-4ddf-92c7-e4bb3753e595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3324661f420412ba23f9a9bd16fc842, user_id=cab064e21e0745f79ea217b3bcf1ffc1, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:48.885863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f969ab-927c-4ddf-92c7-e4bb3753e595 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:48.955208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1258/2488] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:58:48 2022] PUT /identity/v3/projects/a3324661f420412ba23f9a9bd16fc842/users/cab064e21e0745f79ea217b3bcf1ffc1/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:48.959814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68480618-606d-45b4-9ebf-49388a476f9f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:48.970133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68480618-606d-45b4-9ebf-49388a476f9f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:48.974113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:49.029116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a422e-0cfe-48ba-8b60-9917e5b4f6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:49.029721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a422e-0cfe-48ba-8b60-9917e5b4f6cb admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:49.030007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a422e-0cfe-48ba-8b60-9917e5b4f6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:49.030270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a422e-0cfe-48ba-8b60-9917e5b4f6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:49.030897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a422e-0cfe-48ba-8b60-9917e5b4f6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:49.032473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a422e-0cfe-48ba-8b60-9917e5b4f6cb admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:49.042760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1231/2489] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:58:49.047026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f969ab-927c-4ddf-92c7-e4bb3753e595 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:49.057516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f969ab-927c-4ddf-92c7-e4bb3753e595 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:49.062085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af057fd6-2e54-4c63-881a-6ff15e54e6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:49.062755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af057fd6-2e54-4c63-881a-6ff15e54e6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:49.063034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af057fd6-2e54-4c63-881a-6ff15e54e6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:49.063305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af057fd6-2e54-4c63-881a-6ff15e54e6fd admin admin] PATH_INFO: `/v3/projects/a3324661f420412ba23f9a9bd16fc842/users/cab064e21e0745f79ea217b3bcf1ffc1/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:49.065248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af057fd6-2e54-4c63-881a-6ff15e54e6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3324661f420412ba23f9a9bd16fc842, user_id=cab064e21e0745f79ea217b3bcf1ffc1, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:49.068562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af057fd6-2e54-4c63-881a-6ff15e54e6fd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:49.078658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1259/2490] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:58:49 2022] PUT /identity/v3/projects/a3324661f420412ba23f9a9bd16fc842/users/cab064e21e0745f79ea217b3bcf1ffc1/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:49.089855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897874-87a0-40c4-9c1e-f19867e751c4 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:49.090165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897874-87a0-40c4-9c1e-f19867e751c4 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:49.090441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897874-87a0-40c4-9c1e-f19867e751c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:49.118951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-90897874-87a0-40c4-9c1e-f19867e751c4 None None] MFA Rules not processed for user `cab064e21e0745f79ea217b3bcf1ffc1`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:58:49.120154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-90897874-87a0-40c4-9c1e-f19867e751c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:49.188797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1232/2491] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:58:49 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:58:49.200494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af057fd6-2e54-4c63-881a-6ff15e54e6fd admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:49.202250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-af057fd6-2e54-4c63-881a-6ff15e54e6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:49.224666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:49.261307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af057fd6-2e54-4c63-881a-6ff15e54e6fd admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:49.264902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3a395-5b17-4855-973e-35e3f3c6469f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:49.265533 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3a395-5b17-4855-973e-35e3f3c6469f service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:49.265810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3a395-5b17-4855-973e-35e3f3c6469f service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:49.266079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3a395-5b17-4855-973e-35e3f3c6469f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:49.275105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3a395-5b17-4855-973e-35e3f3c6469f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:49.276523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3a395-5b17-4855-973e-35e3f3c6469f service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:49.286230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1260/2492] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:58:49 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:58:49.430925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90897874-87a0-40c4-9c1e-f19867e751c4 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:49.497527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90897874-87a0-40c4-9c1e-f19867e751c4 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:49.502120 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:49.502808 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:49.503059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:49.503333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] PATH_INFO: `/v3/users/4626b77e9aaf4335b030b4c92c64afad` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:49.505352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4626b77e9aaf4335b030b4c92c64afad)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:49.507271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:49.647738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:49.661429 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4626b77e9aaf4335b030b4c92c64afad'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:49.662081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4626b77e9aaf4335b030b4c92c64afad'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:49.662688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4626b77e9aaf4335b030b4c92c64afad'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:49.674765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4626b77e9aaf4335b030b4c92c64afad'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:49.682513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4626b77e9aaf4335b030b4c92c64afad'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:49.760986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1233/2493] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:58:49 2022] DELETE /identity/v3/users/4626b77e9aaf4335b030b4c92c64afad => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:49.765498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea96763-d784-4b6b-89e0-b07592d358ab None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:49.766002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea96763-d784-4b6b-89e0-b07592d358ab None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:49.766113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea96763-d784-4b6b-89e0-b07592d358ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:49.784867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-3ea96763-d784-4b6b-89e0-b07592d358ab None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:58:49.786783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3ea96763-d784-4b6b-89e0-b07592d358ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:49.845666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1261/2494] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:58:49.856163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:49.857653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:49.918397 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf467f-88c2-425e-97e2-8fa4558bf027 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:49.922039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c64776-31d3-476d-9172-7b33b5126013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:49.922647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64776-31d3-476d-9172-7b33b5126013 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:49.922933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64776-31d3-476d-9172-7b33b5126013 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:49.923206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c64776-31d3-476d-9172-7b33b5126013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:49.932657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c64776-31d3-476d-9172-7b33b5126013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:49.934135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c64776-31d3-476d-9172-7b33b5126013 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:49.943730 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1234/2495] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:58:50.383981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea96763-d784-4b6b-89e0-b07592d358ab None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:50.386307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3ea96763-d784-4b6b-89e0-b07592d358ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:50.398219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea96763-d784-4b6b-89e0-b07592d358ab None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:50.402842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:50.403488 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:50.403767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:50.404165 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] PATH_INFO: `/v3/projects/ee44fc6001a5478fb48c2ed545ec81d2` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:50.405318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee44fc6001a5478fb48c2ed545ec81d2)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:50.407253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:50.537183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee44fc6001a5478fb48c2ed545ec81d2', 'request_id': 'req-766fb458-2062-4fb3-a9ce-37ccb12999fd'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:50.543625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee44fc6001a5478fb48c2ed545ec81d2', 'request_id': 'req-766fb458-2062-4fb3-a9ce-37ccb12999fd'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:50.610236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee44fc6001a5478fb48c2ed545ec81d2', 'request_id': 'req-766fb458-2062-4fb3-a9ce-37ccb12999fd'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:50.622788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] The token cache is being invalidate because project ee44fc6001a5478fb48c2ed545ec81d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:58:50.623218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:50.623956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:50.625730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1262/2496] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:58:50 2022] DELETE /identity/v3/projects/ee44fc6001a5478fb48c2ed545ec81d2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:50.630133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c64776-31d3-476d-9172-7b33b5126013 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:50.639940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c64776-31d3-476d-9172-7b33b5126013 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:50.704293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:50.705430 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:50.705939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:50.706444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] PATH_INFO: `/v3/users/cab064e21e0745f79ea217b3bcf1ffc1` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:50.708973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cab064e21e0745f79ea217b3bcf1ffc1)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:50.711324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:50.918560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cab064e21e0745f79ea217b3bcf1ffc1'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:50.919174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cab064e21e0745f79ea217b3bcf1ffc1'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:50.919828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cab064e21e0745f79ea217b3bcf1ffc1'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:50.924482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:50.932562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cab064e21e0745f79ea217b3bcf1ffc1'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:50.939455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6309fd51-4e11-4a75-b196-0c4e883d5d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cab064e21e0745f79ea217b3bcf1ffc1'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:51.000663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1235/2497] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:58:50 2022] DELETE /identity/v3/users/cab064e21e0745f79ea217b3bcf1ffc1 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:58:51.390944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:51.392632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:51.407810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fb458-2062-4fb3-a9ce-37ccb12999fd admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:51.454523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:51.467508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:51.468143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:51.468820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:51.468820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] PATH_INFO: `/v3/projects/a3324661f420412ba23f9a9bd16fc842` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:51.469945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3324661f420412ba23f9a9bd16fc842)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:51.471830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:51.602139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3324661f420412ba23f9a9bd16fc842', 'request_id': 'req-fde85c40-0703-476c-97ed-d0df7f8432a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:51.611192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3324661f420412ba23f9a9bd16fc842', 'request_id': 'req-fde85c40-0703-476c-97ed-d0df7f8432a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:51.784354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3324661f420412ba23f9a9bd16fc842', 'request_id': 'req-fde85c40-0703-476c-97ed-d0df7f8432a7'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:51.796262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] The token cache is being invalidate because project a3324661f420412ba23f9a9bd16fc842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:58:51.796555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:51.797268 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-fde85c40-0703-476c-97ed-d0df7f8432a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:58:51.798763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1263/2498] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:58:51 2022] DELETE /identity/v3/projects/a3324661f420412ba23f9a9bd16fc842 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:51.804344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39657c2b-60f6-48c1-9ad0-8a8fbeb42576 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:51.804848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39657c2b-60f6-48c1-9ad0-8a8fbeb42576 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:51.804956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39657c2b-60f6-48c1-9ad0-8a8fbeb42576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:51.824202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-39657c2b-60f6-48c1-9ad0-8a8fbeb42576 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:58:51.825415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-39657c2b-60f6-48c1-9ad0-8a8fbeb42576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:51.884808 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1236/2499] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:58:51.892626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f1eca-9e79-433a-bf08-fa5da03757d3 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:51.892896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f1eca-9e79-433a-bf08-fa5da03757d3 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:51.893201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f1eca-9e79-433a-bf08-fa5da03757d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:51.914194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-971f1eca-9e79-433a-bf08-fa5da03757d3 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:58:51.918391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-971f1eca-9e79-433a-bf08-fa5da03757d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:51.923652 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1264/2500] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:58:51.928625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39657c2b-60f6-48c1-9ad0-8a8fbeb42576 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:51.938606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39657c2b-60f6-48c1-9ad0-8a8fbeb42576 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:51.943495 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95741fc-dab7-48af-b59b-62020f1eede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:51.944098 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95741fc-dab7-48af-b59b-62020f1eede4 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:51.944382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95741fc-dab7-48af-b59b-62020f1eede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:51.944686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95741fc-dab7-48af-b59b-62020f1eede4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:51.945566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95741fc-dab7-48af-b59b-62020f1eede4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:58:51.948946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95741fc-dab7-48af-b59b-62020f1eede4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:51.948946 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95741fc-dab7-48af-b59b-62020f1eede4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:51.959581 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1237/2501] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:58:51.964171 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f1eca-9e79-433a-bf08-fa5da03757d3 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:51.974833 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f1eca-9e79-433a-bf08-fa5da03757d3 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:51.979623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45466e6-e386-4f6e-9d8c-9ceae9b2a22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:51.980240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45466e6-e386-4f6e-9d8c-9ceae9b2a22b admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:51.980626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45466e6-e386-4f6e-9d8c-9ceae9b2a22b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:51.980901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45466e6-e386-4f6e-9d8c-9ceae9b2a22b admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:51.981622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45466e6-e386-4f6e-9d8c-9ceae9b2a22b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:51.983506 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45466e6-e386-4f6e-9d8c-9ceae9b2a22b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:52.059318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1265/2502] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:58:51 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:58:52.063670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95741fc-dab7-48af-b59b-62020f1eede4 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:52.074256 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95741fc-dab7-48af-b59b-62020f1eede4 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:52.131337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a9895-b484-4d66-93ad-5a6a9f2c9a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:52.131933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9895-b484-4d66-93ad-5a6a9f2c9a9e admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:52.132211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9895-b484-4d66-93ad-5a6a9f2c9a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:52.132482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9895-b484-4d66-93ad-5a6a9f2c9a9e admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:52.133140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a9895-b484-4d66-93ad-5a6a9f2c9a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:52.135031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a9895-b484-4d66-93ad-5a6a9f2c9a9e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:52.241180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1238/2503] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:58:52 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:58:52.245584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45466e6-e386-4f6e-9d8c-9ceae9b2a22b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:52.256041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45466e6-e386-4f6e-9d8c-9ceae9b2a22b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:52.260625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d18a5b-4205-4a6e-ab36-2cbca13a4a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:52.261289 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d18a5b-4205-4a6e-ab36-2cbca13a4a42 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:52.261571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d18a5b-4205-4a6e-ab36-2cbca13a4a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:52.261844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d18a5b-4205-4a6e-ab36-2cbca13a4a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:52.262478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d18a5b-4205-4a6e-ab36-2cbca13a4a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:52.264057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d18a5b-4205-4a6e-ab36-2cbca13a4a42 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:52.274981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1266/2504] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:58:52.278779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a9895-b484-4d66-93ad-5a6a9f2c9a9e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:52.289397 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a9895-b484-4d66-93ad-5a6a9f2c9a9e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:52.294063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a4d50-171e-4999-87f8-0c59b941e155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:52.294658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a4d50-171e-4999-87f8-0c59b941e155 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:52.294940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a4d50-171e-4999-87f8-0c59b941e155 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:52.295216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a4d50-171e-4999-87f8-0c59b941e155 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:52.295855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a4d50-171e-4999-87f8-0c59b941e155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:52.297482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a4d50-171e-4999-87f8-0c59b941e155 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:52.309108 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1239/2505] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:58:52.322135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d18a5b-4205-4a6e-ab36-2cbca13a4a42 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:52.323632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d18a5b-4205-4a6e-ab36-2cbca13a4a42 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:52.328764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aee1bc-eadf-4415-94da-ae08d0ff4bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:52.329459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aee1bc-eadf-4415-94da-ae08d0ff4bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:52.329760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aee1bc-eadf-4415-94da-ae08d0ff4bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:52.330037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aee1bc-eadf-4415-94da-ae08d0ff4bdf admin admin] PATH_INFO: `/v3/projects/4fc3b649ea7b4bc2b2a251cef7420ed5/users/9e2e8931459e46b2888d4e4c68a9f42b/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:52.345651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aee1bc-eadf-4415-94da-ae08d0ff4bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc3b649ea7b4bc2b2a251cef7420ed5, user_id=9e2e8931459e46b2888d4e4c68a9f42b, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:52.348881 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aee1bc-eadf-4415-94da-ae08d0ff4bdf admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:52.416410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1267/2506] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:58:52 2022] PUT /identity/v3/projects/4fc3b649ea7b4bc2b2a251cef7420ed5/users/9e2e8931459e46b2888d4e4c68a9f42b/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:52.427971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2821a-2dcd-4451-8e54-4faea8b53bb0 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:52.428261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2821a-2dcd-4451-8e54-4faea8b53bb0 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:52.428538 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2821a-2dcd-4451-8e54-4faea8b53bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:52.456515 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-1bf2821a-2dcd-4451-8e54-4faea8b53bb0 None None] MFA Rules not processed for user `9e2e8931459e46b2888d4e4c68a9f42b`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:58:52.458844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1bf2821a-2dcd-4451-8e54-4faea8b53bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:52.483513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:52.525395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1240/2507] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:58:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:58:52.529967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aee1bc-eadf-4415-94da-ae08d0ff4bdf admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:52.592861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aee1bc-eadf-4415-94da-ae08d0ff4bdf admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:52.597669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247758d7-736b-4b68-be63-2368636b3b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:52.598290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247758d7-736b-4b68-be63-2368636b3b1c admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:52.598574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247758d7-736b-4b68-be63-2368636b3b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:52.598852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247758d7-736b-4b68-be63-2368636b3b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:52.599487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247758d7-736b-4b68-be63-2368636b3b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:52.601435 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247758d7-736b-4b68-be63-2368636b3b1c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:52.684767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:58:52.692162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1268/2508] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:58:52 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:58:52.696400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2821a-2dcd-4451-8e54-4faea8b53bb0 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:52.707667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2821a-2dcd-4451-8e54-4faea8b53bb0 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:52.770150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf2084-3a02-4591-8a43-74e7df68ad38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:52.770762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf2084-3a02-4591-8a43-74e7df68ad38 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:52.771095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf2084-3a02-4591-8a43-74e7df68ad38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:52.771318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf2084-3a02-4591-8a43-74e7df68ad38 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:52.771941 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf2084-3a02-4591-8a43-74e7df68ad38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:52.773895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf2084-3a02-4591-8a43-74e7df68ad38 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:52.885421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1241/2509] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:58:52 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:58:52.889508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247758d7-736b-4b68-be63-2368636b3b1c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:52.900587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247758d7-736b-4b68-be63-2368636b3b1c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:52.905319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475036e-8bee-4351-a758-c058724ec709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:52.905927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475036e-8bee-4351-a758-c058724ec709 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:52.906213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475036e-8bee-4351-a758-c058724ec709 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:52.906490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475036e-8bee-4351-a758-c058724ec709 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:52.907129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475036e-8bee-4351-a758-c058724ec709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:52.908778 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475036e-8bee-4351-a758-c058724ec709 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:52.919411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1269/2510] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:58:52.923505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf2084-3a02-4591-8a43-74e7df68ad38 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:52.934382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf2084-3a02-4591-8a43-74e7df68ad38 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:52.938988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822db84-6997-443f-9780-000506afd44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:52.939629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822db84-6997-443f-9780-000506afd44c admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:52.939909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822db84-6997-443f-9780-000506afd44c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:52.940183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822db84-6997-443f-9780-000506afd44c admin admin] PATH_INFO: `/v3/projects/1ce066879caa442d9ea94d4dcc913c2b/users/dc2507e38d3c4ef1bc32fd133adca903/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:52.955794 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822db84-6997-443f-9780-000506afd44c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce066879caa442d9ea94d4dcc913c2b, user_id=dc2507e38d3c4ef1bc32fd133adca903, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:52.958468 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822db84-6997-443f-9780-000506afd44c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:53.029134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1242/2511] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:58:52 2022] PUT /identity/v3/projects/1ce066879caa442d9ea94d4dcc913c2b/users/dc2507e38d3c4ef1bc32fd133adca903/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:53.033081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475036e-8bee-4351-a758-c058724ec709 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:53.043280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475036e-8bee-4351-a758-c058724ec709 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:53.099042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb9099-062b-4ecd-adc6-78e0bbe91d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:53.099651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb9099-062b-4ecd-adc6-78e0bbe91d97 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:53.099932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb9099-062b-4ecd-adc6-78e0bbe91d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:53.100208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb9099-062b-4ecd-adc6-78e0bbe91d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:53.100847 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb9099-062b-4ecd-adc6-78e0bbe91d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:53.102466 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb9099-062b-4ecd-adc6-78e0bbe91d97 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:53.112069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1270/2512] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:58:53.116800 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822db84-6997-443f-9780-000506afd44c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:53.127310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822db84-6997-443f-9780-000506afd44c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:53.132819 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06bab0-2144-43f0-ac7a-945ecb6b9c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:53.133493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06bab0-2144-43f0-ac7a-945ecb6b9c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:53.136386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06bab0-2144-43f0-ac7a-945ecb6b9c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:53.136386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06bab0-2144-43f0-ac7a-945ecb6b9c43 admin admin] PATH_INFO: `/v3/projects/1ce066879caa442d9ea94d4dcc913c2b/users/dc2507e38d3c4ef1bc32fd133adca903/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:53.136386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06bab0-2144-43f0-ac7a-945ecb6b9c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce066879caa442d9ea94d4dcc913c2b, user_id=dc2507e38d3c4ef1bc32fd133adca903, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:53.138565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06bab0-2144-43f0-ac7a-945ecb6b9c43 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:53.148949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1243/2513] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:58:53 2022] PUT /identity/v3/projects/1ce066879caa442d9ea94d4dcc913c2b/users/dc2507e38d3c4ef1bc32fd133adca903/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:58:53.159030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eb723-af9a-4dc7-b4fe-bdcdd949e8f7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:53.159332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eb723-af9a-4dc7-b4fe-bdcdd949e8f7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:53.159604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eb723-af9a-4dc7-b4fe-bdcdd949e8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:53.188500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-2c3eb723-af9a-4dc7-b4fe-bdcdd949e8f7 None None] MFA Rules not processed for user `dc2507e38d3c4ef1bc32fd133adca903`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:58:53.189723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2c3eb723-af9a-4dc7-b4fe-bdcdd949e8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:53.258241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1271/2514] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:58:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:58:53.269385 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06bab0-2144-43f0-ac7a-945ecb6b9c43 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:53.271164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6a06bab0-2144-43f0-ac7a-945ecb6b9c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:58:53.328665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06bab0-2144-43f0-ac7a-945ecb6b9c43 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:53.332601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96418dcc-1e48-48b4-872f-316a6e182944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:53.333227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96418dcc-1e48-48b4-872f-316a6e182944 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:53.333506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96418dcc-1e48-48b4-872f-316a6e182944 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:53.333778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96418dcc-1e48-48b4-872f-316a6e182944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:53.342916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96418dcc-1e48-48b4-872f-316a6e182944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:53.344326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96418dcc-1e48-48b4-872f-316a6e182944 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:53.354210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1244/2515] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:58:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:58:53.529316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3eb723-af9a-4dc7-b4fe-bdcdd949e8f7 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:58:53.539855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3eb723-af9a-4dc7-b4fe-bdcdd949e8f7 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:58:53.543517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cdd7d-b053-48a5-84d0-da05ca1b130c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:58:53.544121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cdd7d-b053-48a5-84d0-da05ca1b130c service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:58:53.544402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cdd7d-b053-48a5-84d0-da05ca1b130c service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:58:53.544678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cdd7d-b053-48a5-84d0-da05ca1b130c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:58:53.602731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cdd7d-b053-48a5-84d0-da05ca1b130c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:58:53.604160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cdd7d-b053-48a5-84d0-da05ca1b130c service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:58:53.613438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1272/2516] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:58:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:59:03.350574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:03.609184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:25.895662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96418dcc-1e48-48b4-872f-316a6e182944 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:25.897574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-96418dcc-1e48-48b4-872f-316a6e182944 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:25.981725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96418dcc-1e48-48b4-872f-316a6e182944 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:25.985366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3217fac-e6e0-4db5-9fd6-709b302de53e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:25.985975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3217fac-e6e0-4db5-9fd6-709b302de53e service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:25.986281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3217fac-e6e0-4db5-9fd6-709b302de53e service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:25.986558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3217fac-e6e0-4db5-9fd6-709b302de53e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:25.988292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f3217fac-e6e0-4db5-9fd6-709b302de53e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:26.097653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3217fac-e6e0-4db5-9fd6-709b302de53e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:26.099154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3217fac-e6e0-4db5-9fd6-709b302de53e service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:26.120233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1245/2517] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:59:29.559691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cdd7d-b053-48a5-84d0-da05ca1b130c service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:29.563491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-fa3cdd7d-b053-48a5-84d0-da05ca1b130c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:29.593378 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cdd7d-b053-48a5-84d0-da05ca1b130c service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:29.598413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:29.599064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:29.599350 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:29.599631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] PATH_INFO: `/v3/users/8da410d2ea0740dc8eeea87244fc1408` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:29.600894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da410d2ea0740dc8eeea87244fc1408)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:29.602840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:29.775136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da410d2ea0740dc8eeea87244fc1408'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:29.775793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da410d2ea0740dc8eeea87244fc1408'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:29.776400 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da410d2ea0740dc8eeea87244fc1408'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:29.783669 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da410d2ea0740dc8eeea87244fc1408'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:29.797141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d17f5a9c-17aa-4d29-8ffc-a30d01a1a5fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da410d2ea0740dc8eeea87244fc1408'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:29.875644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1273/2518] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:29 2022] DELETE /identity/v3/users/8da410d2ea0740dc8eeea87244fc1408 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:30.123784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3217fac-e6e0-4db5-9fd6-709b302de53e service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:30.134422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3217fac-e6e0-4db5-9fd6-709b302de53e service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:30.165873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:30.193963 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:30.194601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:30.194902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:30.195170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] PATH_INFO: `/v3/projects/8e876e59e01b4871ad98bc6d30bf396d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:30.196315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e876e59e01b4871ad98bc6d30bf396d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:30.198345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:30.311680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e876e59e01b4871ad98bc6d30bf396d', 'request_id': 'req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:30.318365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e876e59e01b4871ad98bc6d30bf396d', 'request_id': 'req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:30.326197 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e876e59e01b4871ad98bc6d30bf396d', 'request_id': 'req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:30.406995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] The token cache is being invalidate because project 8e876e59e01b4871ad98bc6d30bf396d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:30.407198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:30.407821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b21b1ea1-0e43-4ba7-91ae-95b965d30cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:30.409359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1246/2519] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:30 2022] DELETE /identity/v3/projects/8e876e59e01b4871ad98bc6d30bf396d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:30.415683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc1efb-8505-46e5-86cd-21a425213da4 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:30.415973 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc1efb-8505-46e5-86cd-21a425213da4 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:30.416281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc1efb-8505-46e5-86cd-21a425213da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:30.475847 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-17bc1efb-8505-46e5-86cd-21a425213da4 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:30.477605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-17bc1efb-8505-46e5-86cd-21a425213da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:30.537822 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1274/2520] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:30.545838 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f4773-c360-4e66-8bb0-1cb34342688b None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:30.546203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f4773-c360-4e66-8bb0-1cb34342688b None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:30.546590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f4773-c360-4e66-8bb0-1cb34342688b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:30.571222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ca2f4773-c360-4e66-8bb0-1cb34342688b None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:30.572465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ca2f4773-c360-4e66-8bb0-1cb34342688b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:30.580850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1247/2521] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:30.585722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc1efb-8505-46e5-86cd-21a425213da4 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:30.595569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc1efb-8505-46e5-86cd-21a425213da4 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:30.601443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bd4e7-4f09-4337-bf49-14e216ee5ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:30.602291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bd4e7-4f09-4337-bf49-14e216ee5ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:30.602603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bd4e7-4f09-4337-bf49-14e216ee5ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:30.602877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bd4e7-4f09-4337-bf49-14e216ee5ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:30.603524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bd4e7-4f09-4337-bf49-14e216ee5ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:59:30.603825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bd4e7-4f09-4337-bf49-14e216ee5ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:30.605498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bd4e7-4f09-4337-bf49-14e216ee5ce8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:30.617791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1275/2522] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:59:30.622588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f4773-c360-4e66-8bb0-1cb34342688b None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:30.632102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f4773-c360-4e66-8bb0-1cb34342688b None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:30.636864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0d36b-4dcd-4bf3-b5f9-b110953f92c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:30.637498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0d36b-4dcd-4bf3-b5f9-b110953f92c6 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:30.637778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0d36b-4dcd-4bf3-b5f9-b110953f92c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:30.638051 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0d36b-4dcd-4bf3-b5f9-b110953f92c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:30.638698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0d36b-4dcd-4bf3-b5f9-b110953f92c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:30.640605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0d36b-4dcd-4bf3-b5f9-b110953f92c6 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:30.731779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1248/2523] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:30 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:30.735964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bd4e7-4f09-4337-bf49-14e216ee5ce8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:30.746406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bd4e7-4f09-4337-bf49-14e216ee5ce8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:30.754147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:30.805405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103b8e1-43bb-4a8a-bc82-7889ee474720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:30.806028 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103b8e1-43bb-4a8a-bc82-7889ee474720 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:30.806964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103b8e1-43bb-4a8a-bc82-7889ee474720 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:30.807246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103b8e1-43bb-4a8a-bc82-7889ee474720 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:30.808274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103b8e1-43bb-4a8a-bc82-7889ee474720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:30.810433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103b8e1-43bb-4a8a-bc82-7889ee474720 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:30.921875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1276/2524] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:30 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:30.937613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0d36b-4dcd-4bf3-b5f9-b110953f92c6 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:30.954113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0d36b-4dcd-4bf3-b5f9-b110953f92c6 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:30.966442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f0df0-eee0-4e9e-be24-2a6ea4d6c750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:30.966442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0df0-eee0-4e9e-be24-2a6ea4d6c750 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:30.966442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0df0-eee0-4e9e-be24-2a6ea4d6c750 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:30.966442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0df0-eee0-4e9e-be24-2a6ea4d6c750 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:30.967038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f0df0-eee0-4e9e-be24-2a6ea4d6c750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:30.967038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f0df0-eee0-4e9e-be24-2a6ea4d6c750 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:30.979527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1249/2525] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:30.980739 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103b8e1-43bb-4a8a-bc82-7889ee474720 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:30.991508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103b8e1-43bb-4a8a-bc82-7889ee474720 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:31.000109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defbba5-43bb-4aa3-826a-773f04c64729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:31.000730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defbba5-43bb-4aa3-826a-773f04c64729 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:31.001174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defbba5-43bb-4aa3-826a-773f04c64729 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:31.001455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defbba5-43bb-4aa3-826a-773f04c64729 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:31.002107 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defbba5-43bb-4aa3-826a-773f04c64729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:31.003700 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defbba5-43bb-4aa3-826a-773f04c64729 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:31.015934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1277/2526] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:31.025375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f0df0-eee0-4e9e-be24-2a6ea4d6c750 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:31.035492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f0df0-eee0-4e9e-be24-2a6ea4d6c750 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:31.040266 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd7f56-54b3-4e47-99df-ef27cf0161d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:31.045293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd7f56-54b3-4e47-99df-ef27cf0161d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:31.045293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd7f56-54b3-4e47-99df-ef27cf0161d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:31.045293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd7f56-54b3-4e47-99df-ef27cf0161d9 admin admin] PATH_INFO: `/v3/projects/991c8d86a12d4cb9840af61f7578b71a/users/e6477beaa4554428859660b49ec11119/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:31.057781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd7f56-54b3-4e47-99df-ef27cf0161d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=991c8d86a12d4cb9840af61f7578b71a, user_id=e6477beaa4554428859660b49ec11119, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:31.060446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd7f56-54b3-4e47-99df-ef27cf0161d9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:31.164406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1250/2527] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:31 2022] PUT /identity/v3/projects/991c8d86a12d4cb9840af61f7578b71a/users/e6477beaa4554428859660b49ec11119/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:31.176340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0e3ec-7f99-4228-a32c-2df13904899a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:31.176645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0e3ec-7f99-4228-a32c-2df13904899a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:31.176903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0e3ec-7f99-4228-a32c-2df13904899a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:31.206314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-0ef0e3ec-7f99-4228-a32c-2df13904899a None None] MFA Rules not processed for user `e6477beaa4554428859660b49ec11119`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:31.207451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0ef0e3ec-7f99-4228-a32c-2df13904899a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:31.276800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1278/2528] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:31.281340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd7f56-54b3-4e47-99df-ef27cf0161d9 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:31.348420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd7f56-54b3-4e47-99df-ef27cf0161d9 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:31.353233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bba11-7530-4939-827b-363c486d5acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:31.353842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bba11-7530-4939-827b-363c486d5acd admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:31.354253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bba11-7530-4939-827b-363c486d5acd admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:31.354529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bba11-7530-4939-827b-363c486d5acd admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:31.355217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bba11-7530-4939-827b-363c486d5acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:31.357134 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bba11-7530-4939-827b-363c486d5acd admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:31.361782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:31.485845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1251/2529] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:31 2022] POST /identity/v3/projects => generated 379 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:31.489831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0e3ec-7f99-4228-a32c-2df13904899a None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:31.501370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0e3ec-7f99-4228-a32c-2df13904899a None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:31.559446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84a3a-1c9f-486e-9dcd-2aa6df7f255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:31.560101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84a3a-1c9f-486e-9dcd-2aa6df7f255d admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:31.560376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84a3a-1c9f-486e-9dcd-2aa6df7f255d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:31.560651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84a3a-1c9f-486e-9dcd-2aa6df7f255d admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:31.561301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd84a3a-1c9f-486e-9dcd-2aa6df7f255d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:31.563213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd84a3a-1c9f-486e-9dcd-2aa6df7f255d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:31.675536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1279/2530] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:31 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:31.679671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bba11-7530-4939-827b-363c486d5acd admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:31.690038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bba11-7530-4939-827b-363c486d5acd admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:31.695545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4727dc7-1266-4812-84bb-6332fd23a97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:31.696151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4727dc7-1266-4812-84bb-6332fd23a97d admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:31.696454 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4727dc7-1266-4812-84bb-6332fd23a97d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:31.696713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4727dc7-1266-4812-84bb-6332fd23a97d admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:31.697438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4727dc7-1266-4812-84bb-6332fd23a97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:31.702323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4727dc7-1266-4812-84bb-6332fd23a97d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:31.709731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1252/2531] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:31.714353 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84a3a-1c9f-486e-9dcd-2aa6df7f255d admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:31.724345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84a3a-1c9f-486e-9dcd-2aa6df7f255d admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:31.729366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce68a1c-500b-4bc4-b5ce-a8d62cf07a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:31.730980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce68a1c-500b-4bc4-b5ce-a8d62cf07a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:31.730980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce68a1c-500b-4bc4-b5ce-a8d62cf07a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:31.731297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce68a1c-500b-4bc4-b5ce-a8d62cf07a74 admin admin] PATH_INFO: `/v3/projects/bdb755c6b99a4eba84a5b4623e1103a9/users/c092863b244742b5a1ef94607d66f67e/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:31.735484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:31.759218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce68a1c-500b-4bc4-b5ce-a8d62cf07a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb755c6b99a4eba84a5b4623e1103a9, user_id=c092863b244742b5a1ef94607d66f67e, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:31.759418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce68a1c-500b-4bc4-b5ce-a8d62cf07a74 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:31.836042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1280/2532] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:31 2022] PUT /identity/v3/projects/bdb755c6b99a4eba84a5b4623e1103a9/users/c092863b244742b5a1ef94607d66f67e/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:31.841608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4727dc7-1266-4812-84bb-6332fd23a97d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:31.858413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4727dc7-1266-4812-84bb-6332fd23a97d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:31.919383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9197d3d-7223-4295-b5dc-64003851e164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:31.919982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197d3d-7223-4295-b5dc-64003851e164 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:31.920263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197d3d-7223-4295-b5dc-64003851e164 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:31.920537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197d3d-7223-4295-b5dc-64003851e164 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:31.921207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9197d3d-7223-4295-b5dc-64003851e164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:31.922802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9197d3d-7223-4295-b5dc-64003851e164 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:31.932855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1253/2533] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:31.937168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce68a1c-500b-4bc4-b5ce-a8d62cf07a74 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:31.947342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce68a1c-500b-4bc4-b5ce-a8d62cf07a74 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:31.951916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71082e-7e7d-4e7a-a026-46ecd7a2a3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:31.952561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71082e-7e7d-4e7a-a026-46ecd7a2a3c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:31.952839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71082e-7e7d-4e7a-a026-46ecd7a2a3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:31.953154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71082e-7e7d-4e7a-a026-46ecd7a2a3c0 admin admin] PATH_INFO: `/v3/projects/bdb755c6b99a4eba84a5b4623e1103a9/users/c092863b244742b5a1ef94607d66f67e/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:31.955038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71082e-7e7d-4e7a-a026-46ecd7a2a3c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb755c6b99a4eba84a5b4623e1103a9, user_id=c092863b244742b5a1ef94607d66f67e, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:31.957686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71082e-7e7d-4e7a-a026-46ecd7a2a3c0 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:31.968230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1281/2534] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:31 2022] PUT /identity/v3/projects/bdb755c6b99a4eba84a5b4623e1103a9/users/c092863b244742b5a1ef94607d66f67e/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:31.979070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81dcf-8480-4183-b9c3-0ee856c49674 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:31.979356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81dcf-8480-4183-b9c3-0ee856c49674 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:31.979630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81dcf-8480-4183-b9c3-0ee856c49674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:32.007340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-c0f81dcf-8480-4183-b9c3-0ee856c49674 None None] MFA Rules not processed for user `c092863b244742b5a1ef94607d66f67e`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:32.008535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c0f81dcf-8480-4183-b9c3-0ee856c49674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:32.084355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1254/2535] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:32.096521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71082e-7e7d-4e7a-a026-46ecd7a2a3c0 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:32.098298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3c71082e-7e7d-4e7a-a026-46ecd7a2a3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:32.159679 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71082e-7e7d-4e7a-a026-46ecd7a2a3c0 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:32.163964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ece5a-2738-43f3-9001-64b958f8f874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:32.164575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ece5a-2738-43f3-9001-64b958f8f874 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:32.164857 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ece5a-2738-43f3-9001-64b958f8f874 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:32.165191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ece5a-2738-43f3-9001-64b958f8f874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:32.174744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ece5a-2738-43f3-9001-64b958f8f874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:32.176296 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ece5a-2738-43f3-9001-64b958f8f874 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:32.188692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1282/2536] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:59:33.181532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f81dcf-8480-4183-b9c3-0ee856c49674 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:33.191834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f81dcf-8480-4183-b9c3-0ee856c49674 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:33.196720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24887441-4721-4f90-9cfc-2075803ee134 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c092863b244742b5a1ef94607d66f67e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bdb755c6b99a4eba84a5b4623e1103a9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:33.197352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24887441-4721-4f90-9cfc-2075803ee134 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:33.197633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24887441-4721-4f90-9cfc-2075803ee134 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:33.197910 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24887441-4721-4f90-9cfc-2075803ee134 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:33.198557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24887441-4721-4f90-9cfc-2075803ee134 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:59:33.198860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24887441-4721-4f90-9cfc-2075803ee134 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:33.200435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24887441-4721-4f90-9cfc-2075803ee134 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:33.211948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1255/2537] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:59:33.216023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ece5a-2738-43f3-9001-64b958f8f874 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:33.225741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ece5a-2738-43f3-9001-64b958f8f874 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:33.230398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55c3a5-ad3d-4fbf-ac62-23f327b2352c tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c092863b244742b5a1ef94607d66f67e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bdb755c6b99a4eba84a5b4623e1103a9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:33.231032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55c3a5-ad3d-4fbf-ac62-23f327b2352c tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:33.231313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55c3a5-ad3d-4fbf-ac62-23f327b2352c tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:33.231585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55c3a5-ad3d-4fbf-ac62-23f327b2352c tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:33.232211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55c3a5-ad3d-4fbf-ac62-23f327b2352c tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:33.234131 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55c3a5-ad3d-4fbf-ac62-23f327b2352c tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:33.335722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1283/2538] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:33 2022] POST /identity/v3/projects => generated 354 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:33.340003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24887441-4721-4f90-9cfc-2075803ee134 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:33.350315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24887441-4721-4f90-9cfc-2075803ee134 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:33.393124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:33.409178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5b346-20cf-49db-a255-5b3beb91fd01 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c092863b244742b5a1ef94607d66f67e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bdb755c6b99a4eba84a5b4623e1103a9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:33.409783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5b346-20cf-49db-a255-5b3beb91fd01 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:33.410068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5b346-20cf-49db-a255-5b3beb91fd01 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:33.410342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5b346-20cf-49db-a255-5b3beb91fd01 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:33.410982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5b346-20cf-49db-a255-5b3beb91fd01 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:59:33.411335 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5b346-20cf-49db-a255-5b3beb91fd01 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:33.413240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed5b346-20cf-49db-a255-5b3beb91fd01 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:33.424939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1256/2539] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:59:33.468673 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55c3a5-ad3d-4fbf-ac62-23f327b2352c tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:33.479385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55c3a5-ad3d-4fbf-ac62-23f327b2352c tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:33.484272 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c092863b244742b5a1ef94607d66f67e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bdb755c6b99a4eba84a5b4623e1103a9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:33.484904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:33.485237 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:33.485514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] PATH_INFO: `/v3/projects/bc07e7369d554a108adf57feb6f0228b` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:33.486629 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bc07e7369d554a108adf57feb6f0228b)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:33.488540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:33.648616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc07e7369d554a108adf57feb6f0228b', 'request_id': 'req-624b8b54-41cf-4f9b-a63c-f16231a180d9'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:33.660124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc07e7369d554a108adf57feb6f0228b', 'request_id': 'req-624b8b54-41cf-4f9b-a63c-f16231a180d9'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:33.727126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc07e7369d554a108adf57feb6f0228b', 'request_id': 'req-624b8b54-41cf-4f9b-a63c-f16231a180d9'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:33.738845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] The token cache is being invalidate because project bc07e7369d554a108adf57feb6f0228b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:33.739230 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:33.740609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-624b8b54-41cf-4f9b-a63c-f16231a180d9 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:33.742196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1284/2540] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:33 2022] DELETE /identity/v3/projects/bc07e7369d554a108adf57feb6f0228b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:34.367640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5b346-20cf-49db-a255-5b3beb91fd01 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:34.369725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-eed5b346-20cf-49db-a255-5b3beb91fd01 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:34.440881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5b346-20cf-49db-a255-5b3beb91fd01 tempest-VolumeQuotaClassesTest-346101045 tempest-VolumeQuotaClassesTest-346101045-project-admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:34.498039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:34.498627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:34.498908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:34.499215 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] PATH_INFO: `/v3/users/e6477beaa4554428859660b49ec11119` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:34.500351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6477beaa4554428859660b49ec11119)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:34.502426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:34.710292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6477beaa4554428859660b49ec11119'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:34.710964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6477beaa4554428859660b49ec11119'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:34.711558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6477beaa4554428859660b49ec11119'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:34.724029 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6477beaa4554428859660b49ec11119'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:34.730970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6477beaa4554428859660b49ec11119'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:34.814943 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1257/2541] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:34 2022] DELETE /identity/v3/users/e6477beaa4554428859660b49ec11119 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:34.820321 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764df4a-a79c-4fc2-923e-980b55e74a3e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:34.820832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764df4a-a79c-4fc2-923e-980b55e74a3e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:34.820953 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764df4a-a79c-4fc2-923e-980b55e74a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:34.840403 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d764df4a-a79c-4fc2-923e-980b55e74a3e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:34.841643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d764df4a-a79c-4fc2-923e-980b55e74a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:34.874480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:34.905391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1285/2542] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:34.915351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:34.916870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:34.922659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:34.978130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f8f39-c8b6-42d3-83e8-7a0b34d0f2b6 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:34.981804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0f418-a70c-4b1d-a9a3-83ac14d7475f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:34.982409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f418-a70c-4b1d-a9a3-83ac14d7475f service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:34.982690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f418-a70c-4b1d-a9a3-83ac14d7475f service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:34.982965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f418-a70c-4b1d-a9a3-83ac14d7475f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:34.993378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0f418-a70c-4b1d-a9a3-83ac14d7475f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:35.002138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0f418-a70c-4b1d-a9a3-83ac14d7475f service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:35.004651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1258/2543] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:59:35.442474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764df4a-a79c-4fc2-923e-980b55e74a3e None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:35.444012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d764df4a-a79c-4fc2-923e-980b55e74a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:35.458315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764df4a-a79c-4fc2-923e-980b55e74a3e None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:35.463897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:35.464548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:35.464848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:35.465172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] PATH_INFO: `/v3/projects/991c8d86a12d4cb9840af61f7578b71a` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:35.466293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] RBAC: Authorizing `identity:delete_project(project_id=991c8d86a12d4cb9840af61f7578b71a)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:35.468279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:35.607491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '991c8d86a12d4cb9840af61f7578b71a', 'request_id': 'req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:35.613836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '991c8d86a12d4cb9840af61f7578b71a', 'request_id': 'req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:35.704128 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '991c8d86a12d4cb9840af61f7578b71a', 'request_id': 'req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:35.715870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] The token cache is being invalidate because project 991c8d86a12d4cb9840af61f7578b71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:35.716164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:35.716792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:35.718303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1286/2544] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:35 2022] DELETE /identity/v3/projects/991c8d86a12d4cb9840af61f7578b71a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:35.729661 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0f418-a70c-4b1d-a9a3-83ac14d7475f service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:35.739713 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0f418-a70c-4b1d-a9a3-83ac14d7475f service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:35.800625 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:35.801278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:35.801563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:35.801840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] PATH_INFO: `/v3/users/c092863b244742b5a1ef94607d66f67e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:35.803473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c092863b244742b5a1ef94607d66f67e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:35.805436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:36.015191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c092863b244742b5a1ef94607d66f67e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:36.015944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c092863b244742b5a1ef94607d66f67e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:36.016554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c092863b244742b5a1ef94607d66f67e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:36.029968 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c092863b244742b5a1ef94607d66f67e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:36.037022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-01fef3c0-b8c3-44a7-8e13-a724308314e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c092863b244742b5a1ef94607d66f67e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:36.126549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1259/2545] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:35 2022] DELETE /identity/v3/users/c092863b244742b5a1ef94607d66f67e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:36.570121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:36.570121 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:36.580612 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa14b9a-c127-4dd3-bfe3-0fb7c8fa21af admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:36.641999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:36.642659 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:36.642937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:36.643209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] PATH_INFO: `/v3/projects/bdb755c6b99a4eba84a5b4623e1103a9` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:36.644344 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb755c6b99a4eba84a5b4623e1103a9)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:36.646430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:36.745217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb755c6b99a4eba84a5b4623e1103a9', 'request_id': 'req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:36.751609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb755c6b99a4eba84a5b4623e1103a9', 'request_id': 'req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:36.841128 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb755c6b99a4eba84a5b4623e1103a9', 'request_id': 'req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:36.842455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:36.853314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] The token cache is being invalidate because project bdb755c6b99a4eba84a5b4623e1103a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:36.853611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:36.854238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a4c55639-d24b-4704-a5f4-04e0dbb87ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:36.855722 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1287/2546] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:36 2022] DELETE /identity/v3/projects/bdb755c6b99a4eba84a5b4623e1103a9 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:36.860706 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb2ab5-b600-4fd7-98d4-0c6a1bf38b72 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:36.861186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb2ab5-b600-4fd7-98d4-0c6a1bf38b72 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:36.861281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb2ab5-b600-4fd7-98d4-0c6a1bf38b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:36.882463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-8ceb2ab5-b600-4fd7-98d4-0c6a1bf38b72 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:36.883656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8ceb2ab5-b600-4fd7-98d4-0c6a1bf38b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:36.902470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:36.943605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1260/2547] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:36.951973 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa158fe-5e34-4124-b10b-87cd587bc22c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:36.952477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa158fe-5e34-4124-b10b-87cd587bc22c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:36.953010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa158fe-5e34-4124-b10b-87cd587bc22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:36.977324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-3fa158fe-5e34-4124-b10b-87cd587bc22c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:36.978534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3fa158fe-5e34-4124-b10b-87cd587bc22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:36.986339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1288/2548] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:36.991280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb2ab5-b600-4fd7-98d4-0c6a1bf38b72 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:37.001873 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb2ab5-b600-4fd7-98d4-0c6a1bf38b72 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:37.007407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549f79b-e1a8-4b98-9fef-7b33c9347a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:37.008010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549f79b-e1a8-4b98-9fef-7b33c9347a1c admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:37.008292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549f79b-e1a8-4b98-9fef-7b33c9347a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:37.008566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549f79b-e1a8-4b98-9fef-7b33c9347a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:37.009251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549f79b-e1a8-4b98-9fef-7b33c9347a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:59:37.009557 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549f79b-e1a8-4b98-9fef-7b33c9347a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:37.011149 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549f79b-e1a8-4b98-9fef-7b33c9347a1c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:37.023607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1261/2549] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:59:37.027941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa158fe-5e34-4124-b10b-87cd587bc22c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:37.037254 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa158fe-5e34-4124-b10b-87cd587bc22c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:37.043157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86698cb-3326-423c-8c49-ec1b2071dd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:37.043157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86698cb-3326-423c-8c49-ec1b2071dd28 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:37.043157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86698cb-3326-423c-8c49-ec1b2071dd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:37.043157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86698cb-3326-423c-8c49-ec1b2071dd28 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:37.043579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86698cb-3326-423c-8c49-ec1b2071dd28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:37.045502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86698cb-3326-423c-8c49-ec1b2071dd28 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:37.137592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1289/2550] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:37 2022] POST /identity/v3/projects => generated 399 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:37.142844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549f79b-e1a8-4b98-9fef-7b33c9347a1c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:37.153135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549f79b-e1a8-4b98-9fef-7b33c9347a1c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:37.212399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38cbf0-4d12-4068-b1f5-b66b79870269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:37.213005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38cbf0-4d12-4068-b1f5-b66b79870269 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:37.213311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38cbf0-4d12-4068-b1f5-b66b79870269 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:37.213583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38cbf0-4d12-4068-b1f5-b66b79870269 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:37.214205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38cbf0-4d12-4068-b1f5-b66b79870269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:37.216117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38cbf0-4d12-4068-b1f5-b66b79870269 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:37.301846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1262/2551] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:37 2022] POST /identity/v3/users => generated 300 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:37.312284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86698cb-3326-423c-8c49-ec1b2071dd28 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:37.322990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86698cb-3326-423c-8c49-ec1b2071dd28 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:37.327881 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa980-485b-4f2d-96ce-1a10fd01fa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:37.329947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa980-485b-4f2d-96ce-1a10fd01fa25 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:37.329947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa980-485b-4f2d-96ce-1a10fd01fa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:37.329947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa980-485b-4f2d-96ce-1a10fd01fa25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:37.329947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fa980-485b-4f2d-96ce-1a10fd01fa25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:37.331416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fa980-485b-4f2d-96ce-1a10fd01fa25 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:37.343210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1290/2552] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:37.347283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38cbf0-4d12-4068-b1f5-b66b79870269 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:37.359111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38cbf0-4d12-4068-b1f5-b66b79870269 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:37.363924 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71de3c-ecc4-4c07-80b1-1bf99370710a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:37.364529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71de3c-ecc4-4c07-80b1-1bf99370710a admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:37.364853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71de3c-ecc4-4c07-80b1-1bf99370710a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:37.365161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71de3c-ecc4-4c07-80b1-1bf99370710a admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:37.365925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71de3c-ecc4-4c07-80b1-1bf99370710a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:37.367507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71de3c-ecc4-4c07-80b1-1bf99370710a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:37.378630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1263/2553] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:37.382973 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa980-485b-4f2d-96ce-1a10fd01fa25 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:37.393927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa980-485b-4f2d-96ce-1a10fd01fa25 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:37.399244 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6af11-d832-4617-88b0-7e9c03ff2b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:37.400000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6af11-d832-4617-88b0-7e9c03ff2b24 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:37.400192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6af11-d832-4617-88b0-7e9c03ff2b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:37.400469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6af11-d832-4617-88b0-7e9c03ff2b24 admin admin] PATH_INFO: `/v3/projects/12b38997bb6d45e1a076998b18284b53/users/60ebf8194f4d49b7bea22243dd2a325d/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:37.417464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6af11-d832-4617-88b0-7e9c03ff2b24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b38997bb6d45e1a076998b18284b53, user_id=60ebf8194f4d49b7bea22243dd2a325d, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:37.420162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6af11-d832-4617-88b0-7e9c03ff2b24 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:37.490719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1291/2554] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:37 2022] PUT /identity/v3/projects/12b38997bb6d45e1a076998b18284b53/users/60ebf8194f4d49b7bea22243dd2a325d/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:37.502287 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60afc1c-4abd-4206-a31f-b7b131188f6d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:37.502568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60afc1c-4abd-4206-a31f-b7b131188f6d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:37.502837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60afc1c-4abd-4206-a31f-b7b131188f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:37.534124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a60afc1c-4abd-4206-a31f-b7b131188f6d None None] MFA Rules not processed for user `60ebf8194f4d49b7bea22243dd2a325d`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:37.535471 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a60afc1c-4abd-4206-a31f-b7b131188f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:37.611201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1264/2555] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:37 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:37.612239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6af11-d832-4617-88b0-7e9c03ff2b24 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:37.682075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6af11-d832-4617-88b0-7e9c03ff2b24 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:37.687077 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e5f01-7d77-47b7-ac97-fac3f259c6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:37.687796 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5f01-7d77-47b7-ac97-fac3f259c6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:37.688178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5f01-7d77-47b7-ac97-fac3f259c6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:37.688548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5f01-7d77-47b7-ac97-fac3f259c6a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:37.689383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e5f01-7d77-47b7-ac97-fac3f259c6a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:37.691453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e5f01-7d77-47b7-ac97-fac3f259c6a6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:37.787841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1292/2556] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:37 2022] POST /identity/v3/projects => generated 399 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:59:37.790276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60afc1c-4abd-4206-a31f-b7b131188f6d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:37.801031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60afc1c-4abd-4206-a31f-b7b131188f6d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:37.830371 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:37.890978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0281d-46c0-4839-a2d0-1b0c9aefd897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:37.891945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0281d-46c0-4839-a2d0-1b0c9aefd897 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:37.892129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0281d-46c0-4839-a2d0-1b0c9aefd897 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:37.892469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0281d-46c0-4839-a2d0-1b0c9aefd897 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:37.893450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0281d-46c0-4839-a2d0-1b0c9aefd897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:37.895701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0281d-46c0-4839-a2d0-1b0c9aefd897 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:38.002000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1265/2557] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:37 2022] POST /identity/v3/users => generated 306 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:38.008960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e5f01-7d77-47b7-ac97-fac3f259c6a6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:38.016652 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e5f01-7d77-47b7-ac97-fac3f259c6a6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:38.022445 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f25e0-d7cd-4e43-88d2-bcc1e5df84cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:38.023124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f25e0-d7cd-4e43-88d2-bcc1e5df84cb admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:38.023849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f25e0-d7cd-4e43-88d2-bcc1e5df84cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:38.023849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f25e0-d7cd-4e43-88d2-bcc1e5df84cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:38.024560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f25e0-d7cd-4e43-88d2-bcc1e5df84cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:38.026994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f25e0-d7cd-4e43-88d2-bcc1e5df84cb admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:38.041962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1293/2558] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:59:38.043391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0281d-46c0-4839-a2d0-1b0c9aefd897 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:38.054526 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0281d-46c0-4839-a2d0-1b0c9aefd897 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:38.059263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b886-8952-412f-a8e2-8b1a2580149f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:38.059938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b886-8952-412f-a8e2-8b1a2580149f admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:38.060218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b886-8952-412f-a8e2-8b1a2580149f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:38.060497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b886-8952-412f-a8e2-8b1a2580149f admin admin] PATH_INFO: `/v3/projects/93562b0ee47d4ac2afc3d6ca4dd604a0/users/6f3dfef83de24648912131f306fdb07f/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:38.076259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154b886-8952-412f-a8e2-8b1a2580149f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93562b0ee47d4ac2afc3d6ca4dd604a0, user_id=6f3dfef83de24648912131f306fdb07f, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:38.079013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154b886-8952-412f-a8e2-8b1a2580149f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:38.147427 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1266/2559] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:38 2022] PUT /identity/v3/projects/93562b0ee47d4ac2afc3d6ca4dd604a0/users/6f3dfef83de24648912131f306fdb07f/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:38.151654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f25e0-d7cd-4e43-88d2-bcc1e5df84cb admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:38.162593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f25e0-d7cd-4e43-88d2-bcc1e5df84cb admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:38.222974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7448eb-8143-4230-904a-909595cd40d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:38.223591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7448eb-8143-4230-904a-909595cd40d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:38.223887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7448eb-8143-4230-904a-909595cd40d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:38.224672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7448eb-8143-4230-904a-909595cd40d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:38.225719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7448eb-8143-4230-904a-909595cd40d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:38.227308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7448eb-8143-4230-904a-909595cd40d7 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:38.228829 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:38.238266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1294/2560] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:38.244649 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b886-8952-412f-a8e2-8b1a2580149f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:38.255760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b886-8952-412f-a8e2-8b1a2580149f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:38.260579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6101a-ae7d-475c-bc33-8e659615ac9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:38.261288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6101a-ae7d-475c-bc33-8e659615ac9d admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:38.261576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6101a-ae7d-475c-bc33-8e659615ac9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:38.261925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6101a-ae7d-475c-bc33-8e659615ac9d admin admin] PATH_INFO: `/v3/projects/93562b0ee47d4ac2afc3d6ca4dd604a0/users/6f3dfef83de24648912131f306fdb07f/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:38.265097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6101a-ae7d-475c-bc33-8e659615ac9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93562b0ee47d4ac2afc3d6ca4dd604a0, user_id=6f3dfef83de24648912131f306fdb07f, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:38.268190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6101a-ae7d-475c-bc33-8e659615ac9d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:38.284250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1267/2561] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:38 2022] PUT /identity/v3/projects/93562b0ee47d4ac2afc3d6ca4dd604a0/users/6f3dfef83de24648912131f306fdb07f/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:38.295009 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff79167-a314-4221-8adc-a1071f23af16 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:38.295329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff79167-a314-4221-8adc-a1071f23af16 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:38.295610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff79167-a314-4221-8adc-a1071f23af16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:38.327840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7ff79167-a314-4221-8adc-a1071f23af16 None None] MFA Rules not processed for user `6f3dfef83de24648912131f306fdb07f`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:38.329161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7ff79167-a314-4221-8adc-a1071f23af16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:38.405561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1295/2562] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:38 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:38.417438 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6101a-ae7d-475c-bc33-8e659615ac9d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:38.420305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f7b6101a-ae7d-475c-bc33-8e659615ac9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:38.483087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6101a-ae7d-475c-bc33-8e659615ac9d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:38.486822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2964e4-4ed0-49ca-95c0-12bdd55a8ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:38.487433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2964e4-4ed0-49ca-95c0-12bdd55a8ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:38.487714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2964e4-4ed0-49ca-95c0-12bdd55a8ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:38.487986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2964e4-4ed0-49ca-95c0-12bdd55a8ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:38.498315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2964e4-4ed0-49ca-95c0-12bdd55a8ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:38.500318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2964e4-4ed0-49ca-95c0-12bdd55a8ed1 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:38.509207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1268/2563] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:38 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:59:38.808210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff79167-a314-4221-8adc-a1071f23af16 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:38.815408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff79167-a314-4221-8adc-a1071f23af16 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:38.819112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed33dc9-9ddb-4254-89f1-c24571a3b1ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:38.819721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed33dc9-9ddb-4254-89f1-c24571a3b1ee service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:38.820006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed33dc9-9ddb-4254-89f1-c24571a3b1ee service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:38.820283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed33dc9-9ddb-4254-89f1-c24571a3b1ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:38.880816 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed33dc9-9ddb-4254-89f1-c24571a3b1ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:38.882604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed33dc9-9ddb-4254-89f1-c24571a3b1ee service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:38.894650 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1296/2564] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:38 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:59:41.576026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2964e4-4ed0-49ca-95c0-12bdd55a8ed1 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:41.577665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bd2964e4-4ed0-49ca-95c0-12bdd55a8ed1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:41.649574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2964e4-4ed0-49ca-95c0-12bdd55a8ed1 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:41.654194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:41.654820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:41.655100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:41.655372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] PATH_INFO: `/v3/users/9e2e8931459e46b2888d4e4c68a9f42b` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:41.656499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e2e8931459e46b2888d4e4c68a9f42b)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:41.658484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:41.674630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:41.826755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e2e8931459e46b2888d4e4c68a9f42b'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:41.827491 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e2e8931459e46b2888d4e4c68a9f42b'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:41.828262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e2e8931459e46b2888d4e4c68a9f42b'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:41.840671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e2e8931459e46b2888d4e4c68a9f42b'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:41.847601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e2e8931459e46b2888d4e4c68a9f42b'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:41.933467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1269/2565] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:41 2022] DELETE /identity/v3/users/9e2e8931459e46b2888d4e4c68a9f42b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:41.938055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f276a55-4dfd-4d17-ab1a-919ec581bda9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:41.938511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f276a55-4dfd-4d17-ab1a-919ec581bda9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:41.938671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f276a55-4dfd-4d17-ab1a-919ec581bda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:41.957845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-5f276a55-4dfd-4d17-ab1a-919ec581bda9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:41.959587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5f276a55-4dfd-4d17-ab1a-919ec581bda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:42.018352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1297/2566] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:42.029416 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:42.030954 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:42.087235 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea537cb1-ef52-453f-b4bc-4343cf92113a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:42.090871 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf00a5-f687-4e7b-9f4f-8e941b6ae3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:42.091464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf00a5-f687-4e7b-9f4f-8e941b6ae3cb service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:42.091741 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf00a5-f687-4e7b-9f4f-8e941b6ae3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:42.092012 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf00a5-f687-4e7b-9f4f-8e941b6ae3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:42.101447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf00a5-f687-4e7b-9f4f-8e941b6ae3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:42.102860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf00a5-f687-4e7b-9f4f-8e941b6ae3cb service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:42.113224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1270/2567] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:59:42.604918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f276a55-4dfd-4d17-ab1a-919ec581bda9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:42.609546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5f276a55-4dfd-4d17-ab1a-919ec581bda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:42.638534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f276a55-4dfd-4d17-ab1a-919ec581bda9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:42.644477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:42.645294 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:42.645672 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:42.646173 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] PATH_INFO: `/v3/projects/4fc3b649ea7b4bc2b2a251cef7420ed5` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:42.650537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc3b649ea7b4bc2b2a251cef7420ed5)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:42.650537 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:42.673922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:42.779071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc3b649ea7b4bc2b2a251cef7420ed5', 'request_id': 'req-d82d1123-b667-432f-a179-dddfaa4a232e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:42.785441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc3b649ea7b4bc2b2a251cef7420ed5', 'request_id': 'req-d82d1123-b667-432f-a179-dddfaa4a232e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:42.867303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc3b649ea7b4bc2b2a251cef7420ed5', 'request_id': 'req-d82d1123-b667-432f-a179-dddfaa4a232e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:42.878938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] The token cache is being invalidate because project 4fc3b649ea7b4bc2b2a251cef7420ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:42.879312 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:42.880026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:42.882176 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1298/2568] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:42 2022] DELETE /identity/v3/projects/4fc3b649ea7b4bc2b2a251cef7420ed5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:42.886210 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf00a5-f687-4e7b-9f4f-8e941b6ae3cb service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:42.896268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf00a5-f687-4e7b-9f4f-8e941b6ae3cb service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:42.967592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:42.968565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:42.968856 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:42.969172 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] PATH_INFO: `/v3/users/dc2507e38d3c4ef1bc32fd133adca903` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:42.970478 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2507e38d3c4ef1bc32fd133adca903)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:42.973274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:43.159689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:43.165872 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2507e38d3c4ef1bc32fd133adca903'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:43.166782 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2507e38d3c4ef1bc32fd133adca903'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:43.167505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2507e38d3c4ef1bc32fd133adca903'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:43.180087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2507e38d3c4ef1bc32fd133adca903'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:43.187473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2dcee0eb-03ea-4019-92c5-814b9631dbb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2507e38d3c4ef1bc32fd133adca903'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:43.278974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1271/2569] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:42 2022] DELETE /identity/v3/users/dc2507e38d3c4ef1bc32fd133adca903 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:43.837831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:43.839562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:43.861256 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d1123-b667-432f-a179-dddfaa4a232e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:43.922876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:43.923520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:43.923797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:43.924069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] PATH_INFO: `/v3/projects/1ce066879caa442d9ea94d4dcc913c2b` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:43.925276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce066879caa442d9ea94d4dcc913c2b)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:43.927207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:44.032539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce066879caa442d9ea94d4dcc913c2b', 'request_id': 'req-42cc2377-7c16-4a27-9208-929ecdb847e1'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:44.040623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce066879caa442d9ea94d4dcc913c2b', 'request_id': 'req-42cc2377-7c16-4a27-9208-929ecdb847e1'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:44.109651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce066879caa442d9ea94d4dcc913c2b', 'request_id': 'req-42cc2377-7c16-4a27-9208-929ecdb847e1'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:44.121609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] The token cache is being invalidate because project 1ce066879caa442d9ea94d4dcc913c2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:44.121902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:44.122539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-42cc2377-7c16-4a27-9208-929ecdb847e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:44.124032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1299/2570] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:43 2022] DELETE /identity/v3/projects/1ce066879caa442d9ea94d4dcc913c2b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:44.129105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a3002-61a4-42f7-b596-c2f2e49646ea None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.129564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a3002-61a4-42f7-b596-c2f2e49646ea None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.129659 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a3002-61a4-42f7-b596-c2f2e49646ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.152595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-bb4a3002-61a4-42f7-b596-c2f2e49646ea None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:44.154002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bb4a3002-61a4-42f7-b596-c2f2e49646ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:44.216417 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1272/2571] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:44.224486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216da874-5075-471d-a42f-280d71c61426 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.224797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216da874-5075-471d-a42f-280d71c61426 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.225098 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216da874-5075-471d-a42f-280d71c61426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.247012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-216da874-5075-471d-a42f-280d71c61426 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:44.248211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-216da874-5075-471d-a42f-280d71c61426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:44.256760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1300/2572] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:44.261222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a3002-61a4-42f7-b596-c2f2e49646ea None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:44.271218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a3002-61a4-42f7-b596-c2f2e49646ea None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:44.276537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17df060-0d2d-40f0-b552-5b37e1befa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:44.279465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17df060-0d2d-40f0-b552-5b37e1befa8a admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.279465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17df060-0d2d-40f0-b552-5b37e1befa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.279465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17df060-0d2d-40f0-b552-5b37e1befa8a admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.279465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17df060-0d2d-40f0-b552-5b37e1befa8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:59:44.279465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17df060-0d2d-40f0-b552-5b37e1befa8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:44.280400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17df060-0d2d-40f0-b552-5b37e1befa8a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:44.292475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1273/2573] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:59:44.296556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216da874-5075-471d-a42f-280d71c61426 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:44.306260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216da874-5075-471d-a42f-280d71c61426 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:44.311241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11005f35-3b05-46ba-a742-6535d7bf41a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:44.311843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11005f35-3b05-46ba-a742-6535d7bf41a5 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.312247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11005f35-3b05-46ba-a742-6535d7bf41a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.312522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11005f35-3b05-46ba-a742-6535d7bf41a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.313192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11005f35-3b05-46ba-a742-6535d7bf41a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:44.315065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11005f35-3b05-46ba-a742-6535d7bf41a5 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:44.321718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17df060-0d2d-40f0-b552-5b37e1befa8a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:44.323519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a17df060-0d2d-40f0-b552-5b37e1befa8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:44.336622 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17df060-0d2d-40f0-b552-5b37e1befa8a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:44.341329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:44.341987 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.342271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.342549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] PATH_INFO: `/v3/users/60ebf8194f4d49b7bea22243dd2a325d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.343688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ebf8194f4d49b7bea22243dd2a325d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:44.345657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:44.411023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1301/2574] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:44 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:59:44.415255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11005f35-3b05-46ba-a742-6535d7bf41a5 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:44.421104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:44.487252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11005f35-3b05-46ba-a742-6535d7bf41a5 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:44.492355 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f41da-661a-44a8-8c81-4c4985951f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:44.493264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f41da-661a-44a8-8c81-4c4985951f05 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.494172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f41da-661a-44a8-8c81-4c4985951f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.494523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f41da-661a-44a8-8c81-4c4985951f05 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.495155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f41da-661a-44a8-8c81-4c4985951f05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:44.497382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f41da-661a-44a8-8c81-4c4985951f05 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:44.558144 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ebf8194f4d49b7bea22243dd2a325d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:44.558737 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ebf8194f4d49b7bea22243dd2a325d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:44.559356 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ebf8194f4d49b7bea22243dd2a325d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:44.571992 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ebf8194f4d49b7bea22243dd2a325d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:44.578902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-e8bdc463-38f4-4fc1-b4b0-56bd018bdf89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ebf8194f4d49b7bea22243dd2a325d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:44.602545 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1302/2575] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:44 2022] POST /identity/v3/users => generated 282 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:59:44.606602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f41da-661a-44a8-8c81-4c4985951f05 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:44.608233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-619f41da-661a-44a8-8c81-4c4985951f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:44.621383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f41da-661a-44a8-8c81-4c4985951f05 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:44.626361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21158f46-10e5-4b91-a97f-281203a902af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:44.626943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21158f46-10e5-4b91-a97f-281203a902af admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.627217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21158f46-10e5-4b91-a97f-281203a902af admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.627485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21158f46-10e5-4b91-a97f-281203a902af admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.630007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21158f46-10e5-4b91-a97f-281203a902af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:44.630007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21158f46-10e5-4b91-a97f-281203a902af admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:44.641951 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1303/2576] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:44.645846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21158f46-10e5-4b91-a97f-281203a902af admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:44.655947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21158f46-10e5-4b91-a97f-281203a902af admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:44.664067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1274/2577] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:44 2022] DELETE /identity/v3/users/60ebf8194f4d49b7bea22243dd2a325d => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:44.664210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d9a40-97b0-4450-8f95-a440a55f59eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:44.664210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d9a40-97b0-4450-8f95-a440a55f59eb admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.664210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d9a40-97b0-4450-8f95-a440a55f59eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.664210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d9a40-97b0-4450-8f95-a440a55f59eb admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.664622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d9a40-97b0-4450-8f95-a440a55f59eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:44.665036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d9a40-97b0-4450-8f95-a440a55f59eb admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:44.667328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9be2e-3ffe-4ba2-b696-90f10b4d76da None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.667692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9be2e-3ffe-4ba2-b696-90f10b4d76da None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.668006 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9be2e-3ffe-4ba2-b696-90f10b4d76da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.679008 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1304/2578] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:59:44.680092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d9a40-97b0-4450-8f95-a440a55f59eb admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:44.688014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a6f9be2e-3ffe-4ba2-b696-90f10b4d76da None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:44.689270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a6f9be2e-3ffe-4ba2-b696-90f10b4d76da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:44.690583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d9a40-97b0-4450-8f95-a440a55f59eb admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:44.699408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:44.749333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67705072-af78-4d34-aaf4-d82247878283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:44.750000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67705072-af78-4d34-aaf4-d82247878283 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.750281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67705072-af78-4d34-aaf4-d82247878283 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.750555 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67705072-af78-4d34-aaf4-d82247878283 admin admin] PATH_INFO: `/v3/projects/97eca4e4dbf44f1ea1bace6ab62a7598/users/0ff8a11f7df24cc6b0c7e65c76179012/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.751607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1275/2579] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:59:44.762665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9be2e-3ffe-4ba2-b696-90f10b4d76da None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:44.764013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a6f9be2e-3ffe-4ba2-b696-90f10b4d76da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:44.766078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67705072-af78-4d34-aaf4-d82247878283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97eca4e4dbf44f1ea1bace6ab62a7598, user_id=0ff8a11f7df24cc6b0c7e65c76179012, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:44.773111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67705072-af78-4d34-aaf4-d82247878283 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:44.843809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9be2e-3ffe-4ba2-b696-90f10b4d76da None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:44.844135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1305/2580] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:44 2022] PUT /identity/v3/projects/97eca4e4dbf44f1ea1bace6ab62a7598/users/0ff8a11f7df24cc6b0c7e65c76179012/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:44.854717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78efec3-0c68-406a-81e3-6e3867bf9d6e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.859562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78efec3-0c68-406a-81e3-6e3867bf9d6e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.859562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78efec3-0c68-406a-81e3-6e3867bf9d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.883998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b78efec3-0c68-406a-81e3-6e3867bf9d6e None None] MFA Rules not processed for user `0ff8a11f7df24cc6b0c7e65c76179012`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:44.885285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b78efec3-0c68-406a-81e3-6e3867bf9d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:44.894083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a820139-0e86-40c0-97ba-bb1def1064d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:44.894700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a820139-0e86-40c0-97ba-bb1def1064d4 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.894999 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a820139-0e86-40c0-97ba-bb1def1064d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.895259 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a820139-0e86-40c0-97ba-bb1def1064d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.930168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:44.950419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1306/2581] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:59:44.954898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78efec3-0c68-406a-81e3-6e3867bf9d6e None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:44.959699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a820139-0e86-40c0-97ba-bb1def1064d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:44.963010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a820139-0e86-40c0-97ba-bb1def1064d4 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:44.966770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78efec3-0c68-406a-81e3-6e3867bf9d6e None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:44.971621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab24d35-1830-46a0-b0e4-74517dabf341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:44.972705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab24d35-1830-46a0-b0e4-74517dabf341 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:44.972977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab24d35-1830-46a0-b0e4-74517dabf341 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:44.973599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab24d35-1830-46a0-b0e4-74517dabf341 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:44.975199 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab24d35-1830-46a0-b0e4-74517dabf341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:44.977282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab24d35-1830-46a0-b0e4-74517dabf341 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:44.977951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1276/2582] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:59:45.069670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1307/2583] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:44 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:59:45.074128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a820139-0e86-40c0-97ba-bb1def1064d4 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:45.079115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:45.085273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a820139-0e86-40c0-97ba-bb1def1064d4 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:45.143740 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4261fc-769e-46ae-ba3d-10aa2f9d00bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:45.144426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4261fc-769e-46ae-ba3d-10aa2f9d00bc admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:45.144543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4261fc-769e-46ae-ba3d-10aa2f9d00bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:45.144803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4261fc-769e-46ae-ba3d-10aa2f9d00bc admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:45.145478 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4261fc-769e-46ae-ba3d-10aa2f9d00bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:45.147779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4261fc-769e-46ae-ba3d-10aa2f9d00bc admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:45.242936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1277/2584] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:45 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:45.247108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab24d35-1830-46a0-b0e4-74517dabf341 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:45.260451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab24d35-1830-46a0-b0e4-74517dabf341 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:45.267273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139b686-71dc-46d2-b85b-891ef9e8fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:45.268243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139b686-71dc-46d2-b85b-891ef9e8fbbb admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:45.268582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139b686-71dc-46d2-b85b-891ef9e8fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:45.268972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139b686-71dc-46d2-b85b-891ef9e8fbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:45.269887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139b686-71dc-46d2-b85b-891ef9e8fbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:45.272476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139b686-71dc-46d2-b85b-891ef9e8fbbb admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:45.285725 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1308/2585] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:45.288564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4261fc-769e-46ae-ba3d-10aa2f9d00bc admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:45.298889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4261fc-769e-46ae-ba3d-10aa2f9d00bc admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:45.303674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66000eb9-9a8c-43c9-b9e4-f4066e03c05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:45.304338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66000eb9-9a8c-43c9-b9e4-f4066e03c05b admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:45.304621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66000eb9-9a8c-43c9-b9e4-f4066e03c05b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:45.305614 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66000eb9-9a8c-43c9-b9e4-f4066e03c05b admin admin] PATH_INFO: `/v3/projects/39dc52bd3e3a44399d44520a564d6bb9/users/a5a5d8d1e3ec4633bac29cd2d95fafdb/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:45.320382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66000eb9-9a8c-43c9-b9e4-f4066e03c05b admin admin] RBAC: Authorizing `identity:create_grant(project_id=39dc52bd3e3a44399d44520a564d6bb9, user_id=a5a5d8d1e3ec4633bac29cd2d95fafdb, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:45.323104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66000eb9-9a8c-43c9-b9e4-f4066e03c05b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:45.424380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1278/2586] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:45 2022] PUT /identity/v3/projects/39dc52bd3e3a44399d44520a564d6bb9/users/a5a5d8d1e3ec4633bac29cd2d95fafdb/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:45.428641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139b686-71dc-46d2-b85b-891ef9e8fbbb admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:45.438632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139b686-71dc-46d2-b85b-891ef9e8fbbb admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:45.460505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66000eb9-9a8c-43c9-b9e4-f4066e03c05b admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:45.462693 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-66000eb9-9a8c-43c9-b9e4-f4066e03c05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:45.503750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35190051-b819-4363-920d-45c3528d0f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:45.504356 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190051-b819-4363-920d-45c3528d0f41 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:45.504634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190051-b819-4363-920d-45c3528d0f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:45.504906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190051-b819-4363-920d-45c3528d0f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:45.505577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35190051-b819-4363-920d-45c3528d0f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:45.507424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35190051-b819-4363-920d-45c3528d0f41 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:45.517606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1309/2587] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:59:45.522298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35190051-b819-4363-920d-45c3528d0f41 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:45.532081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35190051-b819-4363-920d-45c3528d0f41 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:45.535505 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66000eb9-9a8c-43c9-b9e4-f4066e03c05b admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:45.537859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52474b-e822-4b80-8aa9-c22cd3b8094f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:45.538529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52474b-e822-4b80-8aa9-c22cd3b8094f admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:45.538817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52474b-e822-4b80-8aa9-c22cd3b8094f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:45.539096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52474b-e822-4b80-8aa9-c22cd3b8094f admin admin] PATH_INFO: `/v3/projects/39dc52bd3e3a44399d44520a564d6bb9/users/a5a5d8d1e3ec4633bac29cd2d95fafdb/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:45.540994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:45.541163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52474b-e822-4b80-8aa9-c22cd3b8094f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39dc52bd3e3a44399d44520a564d6bb9, user_id=a5a5d8d1e3ec4633bac29cd2d95fafdb, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:45.541647 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:45.541925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:45.542216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] PATH_INFO: `/v3/projects/12b38997bb6d45e1a076998b18284b53` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:45.543340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b38997bb6d45e1a076998b18284b53)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:45.543794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52474b-e822-4b80-8aa9-c22cd3b8094f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:45.545262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:45.555759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1310/2588] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:45 2022] PUT /identity/v3/projects/39dc52bd3e3a44399d44520a564d6bb9/users/a5a5d8d1e3ec4633bac29cd2d95fafdb/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:45.566649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0142a-7279-4176-b54c-dfebc9964d37 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:45.566940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0142a-7279-4176-b54c-dfebc9964d37 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:45.567215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0142a-7279-4176-b54c-dfebc9964d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:45.594969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-48b0142a-7279-4176-b54c-dfebc9964d37 None None] MFA Rules not processed for user `a5a5d8d1e3ec4633bac29cd2d95fafdb`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:45.596222 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-48b0142a-7279-4176-b54c-dfebc9964d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:45.666132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1311/2589] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:59:45.677663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0142a-7279-4176-b54c-dfebc9964d37 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:45.678932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-48b0142a-7279-4176-b54c-dfebc9964d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:45.699317 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b38997bb6d45e1a076998b18284b53', 'request_id': 'req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:45.700634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:45.705804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b38997bb6d45e1a076998b18284b53', 'request_id': 'req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:45.713239 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b38997bb6d45e1a076998b18284b53', 'request_id': 'req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:45.738695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0142a-7279-4176-b54c-dfebc9964d37 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:45.790212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:45.790532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d9805-acfc-41aa-8a6b-0ef756b932fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:45.791253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d9805-acfc-41aa-8a6b-0ef756b932fa service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:45.791613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d9805-acfc-41aa-8a6b-0ef756b932fa service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:45.791887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d9805-acfc-41aa-8a6b-0ef756b932fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:45.794984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] The token cache is being invalidate because project 12b38997bb6d45e1a076998b18284b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:45.798184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:45.798184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:45.798184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1279/2590] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:45 2022] DELETE /identity/v3/projects/12b38997bb6d45e1a076998b18284b53 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:45.801382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:45.802878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:45.854872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d9805-acfc-41aa-8a6b-0ef756b932fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:45.856374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d9805-acfc-41aa-8a6b-0ef756b932fa service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:45.866511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1312/2591] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:59:45.868401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b64e3-8854-4f68-8490-c07ef4ebf3e9 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:45.873450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:45.874075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:45.874351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:45.874622 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] PATH_INFO: `/v3/users/6f3dfef83de24648912131f306fdb07f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:45.875748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f3dfef83de24648912131f306fdb07f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:45.877684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:46.083021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f3dfef83de24648912131f306fdb07f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:46.083700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f3dfef83de24648912131f306fdb07f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:46.085401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f3dfef83de24648912131f306fdb07f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:46.100735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f3dfef83de24648912131f306fdb07f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:46.110874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f3dfef83de24648912131f306fdb07f'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:46.183936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1280/2592] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:45 2022] DELETE /identity/v3/users/6f3dfef83de24648912131f306fdb07f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:46.688151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d9805-acfc-41aa-8a6b-0ef756b932fa service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:46.703786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d9805-acfc-41aa-8a6b-0ef756b932fa service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:46.775027 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:46.775690 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:46.775976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:46.776485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] PATH_INFO: `/v3/projects/93562b0ee47d4ac2afc3d6ca4dd604a0` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:46.777685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=93562b0ee47d4ac2afc3d6ca4dd604a0)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:46.779649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:46.899293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:46.916031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93562b0ee47d4ac2afc3d6ca4dd604a0', 'request_id': 'req-6be561c7-67b7-4a79-9bda-40069f3bf1ba'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:46.924020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93562b0ee47d4ac2afc3d6ca4dd604a0', 'request_id': 'req-6be561c7-67b7-4a79-9bda-40069f3bf1ba'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:46.932556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:46.934118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:46.950883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527eb39-78c4-4297-bee3-31b6dd1c7ae9 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:47.004014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93562b0ee47d4ac2afc3d6ca4dd604a0', 'request_id': 'req-6be561c7-67b7-4a79-9bda-40069f3bf1ba'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:47.016783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:47.016970 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] The token cache is being invalidate because project 93562b0ee47d4ac2afc3d6ca4dd604a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:47.017214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:47.017667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.017781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6be561c7-67b7-4a79-9bda-40069f3bf1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:47.018077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.018423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] PATH_INFO: `/v3/users/0ff8a11f7df24cc6b0c7e65c76179012` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.019209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1313/2593] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:46 2022] DELETE /identity/v3/projects/93562b0ee47d4ac2afc3d6ca4dd604a0 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:47.019702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ff8a11f7df24cc6b0c7e65c76179012)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:47.021850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:47.025692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71c799-2982-4d8b-9448-83fba5e52182 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.025692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71c799-2982-4d8b-9448-83fba5e52182 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.025692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71c799-2982-4d8b-9448-83fba5e52182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.044329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:47.049276 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-bf71c799-2982-4d8b-9448-83fba5e52182 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:47.051122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-bf71c799-2982-4d8b-9448-83fba5e52182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:47.060536 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1314/2594] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:59:47.071024 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f285d-bce3-4084-92e7-b3124133e4a0 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.071548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f285d-bce3-4084-92e7-b3124133e4a0 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.072056 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f285d-bce3-4084-92e7-b3124133e4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.094686 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1b1f285d-bce3-4084-92e7-b3124133e4a0 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:47.095885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1b1f285d-bce3-4084-92e7-b3124133e4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:47.104947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1315/2595] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:59:47.108375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f285d-bce3-4084-92e7-b3124133e4a0 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:47.118131 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f285d-bce3-4084-92e7-b3124133e4a0 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:47.123094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aa50f-e12b-43aa-87b5-28420334408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:47.123861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aa50f-e12b-43aa-87b5-28420334408c admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.124147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aa50f-e12b-43aa-87b5-28420334408c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.124425 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aa50f-e12b-43aa-87b5-28420334408c admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.125103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aa50f-e12b-43aa-87b5-28420334408c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:59:47.125414 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aa50f-e12b-43aa-87b5-28420334408c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:47.127030 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aa50f-e12b-43aa-87b5-28420334408c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:47.143985 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1316/2596] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 18:59:47.149185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aa50f-e12b-43aa-87b5-28420334408c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:47.162512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aa50f-e12b-43aa-87b5-28420334408c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:47.164702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132db3e-e5de-494d-87c4-b15d037c3775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:47.165476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132db3e-e5de-494d-87c4-b15d037c3775 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.165757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132db3e-e5de-494d-87c4-b15d037c3775 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.166025 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132db3e-e5de-494d-87c4-b15d037c3775 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.166645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132db3e-e5de-494d-87c4-b15d037c3775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:47.168551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132db3e-e5de-494d-87c4-b15d037c3775 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:47.252327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ff8a11f7df24cc6b0c7e65c76179012'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:47.252978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ff8a11f7df24cc6b0c7e65c76179012'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:47.253623 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ff8a11f7df24cc6b0c7e65c76179012'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:47.266915 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ff8a11f7df24cc6b0c7e65c76179012'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:47.274858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1317/2597] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:47 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:59:47.275434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-652f20ca-9893-4a55-8af8-d387d0d18dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ff8a11f7df24cc6b0c7e65c76179012'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:47.278900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132db3e-e5de-494d-87c4-b15d037c3775 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:47.280627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7132db3e-e5de-494d-87c4-b15d037c3775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:47.350861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132db3e-e5de-494d-87c4-b15d037c3775 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:47.356122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ab762-2a35-459b-bd99-92bd6a98e958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:47.356783 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ab762-2a35-459b-bd99-92bd6a98e958 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.357091 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ab762-2a35-459b-bd99-92bd6a98e958 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.357359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ab762-2a35-459b-bd99-92bd6a98e958 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.358032 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ab762-2a35-459b-bd99-92bd6a98e958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:47.359426 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1281/2598] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:46 2022] DELETE /identity/v3/users/0ff8a11f7df24cc6b0c7e65c76179012 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:47.362956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ab762-2a35-459b-bd99-92bd6a98e958 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:47.364338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4c55-a436-45ef-a8ef-466310760453 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.364630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4c55-a436-45ef-a8ef-466310760453 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.364909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4c55-a436-45ef-a8ef-466310760453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.386135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b88a4c55-a436-45ef-a8ef-466310760453 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:47.387334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b88a4c55-a436-45ef-a8ef-466310760453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:47.452936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1282/2599] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:59:47.462481 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a4c55-a436-45ef-a8ef-466310760453 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:47.463844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b88a4c55-a436-45ef-a8ef-466310760453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:47.483637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1318/2600] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:47 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:47.488636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ab762-2a35-459b-bd99-92bd6a98e958 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:47.500060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ab762-2a35-459b-bd99-92bd6a98e958 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:47.506712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2f016-2cba-46a9-8d31-a69aea5129be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:47.507322 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2f016-2cba-46a9-8d31-a69aea5129be admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.507620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2f016-2cba-46a9-8d31-a69aea5129be admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.507878 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2f016-2cba-46a9-8d31-a69aea5129be admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.508896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2f016-2cba-46a9-8d31-a69aea5129be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:47.510838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2f016-2cba-46a9-8d31-a69aea5129be admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:47.521186 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1319/2601] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:59:47.523479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a4c55-a436-45ef-a8ef-466310760453 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:47.525527 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2f016-2cba-46a9-8d31-a69aea5129be admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:47.527399 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad9a29-988b-41a8-bc92-8697a8981b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:47.528006 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad9a29-988b-41a8-bc92-8697a8981b0e service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.528289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad9a29-988b-41a8-bc92-8697a8981b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.528564 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad9a29-988b-41a8-bc92-8697a8981b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.535163 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2f016-2cba-46a9-8d31-a69aea5129be admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:47.538482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad9a29-988b-41a8-bc92-8697a8981b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:47.539984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad9a29-988b-41a8-bc92-8697a8981b0e service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:47.541069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7f30d-039f-4efb-9595-ee6dbda371df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:47.541649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f30d-039f-4efb-9595-ee6dbda371df admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.541932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f30d-039f-4efb-9595-ee6dbda371df admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.542206 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f30d-039f-4efb-9595-ee6dbda371df admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.542835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7f30d-039f-4efb-9595-ee6dbda371df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:47.544428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7f30d-039f-4efb-9595-ee6dbda371df admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:47.549716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1283/2602] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:59:47.558689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1320/2603] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:59:47.559798 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad9a29-988b-41a8-bc92-8697a8981b0e service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:47.570705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad9a29-988b-41a8-bc92-8697a8981b0e service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:47.575458 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d952c-b965-4c12-b1bf-b48fe55602a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:47.576133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d952c-b965-4c12-b1bf-b48fe55602a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.576412 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d952c-b965-4c12-b1bf-b48fe55602a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.576687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d952c-b965-4c12-b1bf-b48fe55602a6 admin admin] PATH_INFO: `/v3/projects/588c9c39bd694f1e860679848d754b3c/users/d4b0c5a55bcc40aabec5c9db8cbd31e0/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.592878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1d952c-b965-4c12-b1bf-b48fe55602a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=588c9c39bd694f1e860679848d754b3c, user_id=d4b0c5a55bcc40aabec5c9db8cbd31e0, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:47.595736 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1d952c-b965-4c12-b1bf-b48fe55602a6 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:47.675783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1284/2604] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:47 2022] PUT /identity/v3/projects/588c9c39bd694f1e860679848d754b3c/users/d4b0c5a55bcc40aabec5c9db8cbd31e0/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:47.686152 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8184-e46d-4c67-893d-38c074aee7a3 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.686442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8184-e46d-4c67-893d-38c074aee7a3 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.686751 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8184-e46d-4c67-893d-38c074aee7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.716584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-daad8184-e46d-4c67-893d-38c074aee7a3 None None] MFA Rules not processed for user `d4b0c5a55bcc40aabec5c9db8cbd31e0`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:47.717851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-daad8184-e46d-4c67-893d-38c074aee7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:47.749512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:47.795699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1321/2605] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:47.800194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d952c-b965-4c12-b1bf-b48fe55602a6 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:47.820128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:47.866465 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d952c-b965-4c12-b1bf-b48fe55602a6 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:47.871868 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871a84c-e943-4b23-b785-3435037217d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:47.872475 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871a84c-e943-4b23-b785-3435037217d3 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:47.872759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871a84c-e943-4b23-b785-3435037217d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:47.873032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871a84c-e943-4b23-b785-3435037217d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:47.873691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871a84c-e943-4b23-b785-3435037217d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:47.875603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871a84c-e943-4b23-b785-3435037217d3 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:47.975147 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1285/2606] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:47 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:47.979510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad8184-e46d-4c67-893d-38c074aee7a3 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:47.991155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad8184-e46d-4c67-893d-38c074aee7a3 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:48.003251 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871a84c-e943-4b23-b785-3435037217d3 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:48.005308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0871a84c-e943-4b23-b785-3435037217d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:48.069861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7c60b-ae7d-4ed0-80b1-d5f60431246a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:48.070475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7c60b-ae7d-4ed0-80b1-d5f60431246a admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:48.070750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7c60b-ae7d-4ed0-80b1-d5f60431246a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:48.071012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7c60b-ae7d-4ed0-80b1-d5f60431246a admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:48.071649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7c60b-ae7d-4ed0-80b1-d5f60431246a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:48.073637 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7c60b-ae7d-4ed0-80b1-d5f60431246a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:48.081931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871a84c-e943-4b23-b785-3435037217d3 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:48.086845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:48.092254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:48.092254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:48.092254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] PATH_INFO: `/v3/projects/97eca4e4dbf44f1ea1bace6ab62a7598` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:48.092254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97eca4e4dbf44f1ea1bace6ab62a7598)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:48.092254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:48.187118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1322/2607] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:47 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:48.190826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7c60b-ae7d-4ed0-80b1-d5f60431246a admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:48.200865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7c60b-ae7d-4ed0-80b1-d5f60431246a admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:48.205625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf72bb-1fb2-4c7e-870b-896491c4dca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:48.206206 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf72bb-1fb2-4c7e-870b-896491c4dca4 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:48.206486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf72bb-1fb2-4c7e-870b-896491c4dca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:48.206869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf72bb-1fb2-4c7e-870b-896491c4dca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:48.207521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf72bb-1fb2-4c7e-870b-896491c4dca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:48.209361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf72bb-1fb2-4c7e-870b-896491c4dca4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:48.219675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1323/2608] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:59:48.223596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf72bb-1fb2-4c7e-870b-896491c4dca4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:48.231387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97eca4e4dbf44f1ea1bace6ab62a7598', 'request_id': 'req-0aef55dc-fb3c-440e-af30-c3001461e132'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:48.233907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf72bb-1fb2-4c7e-870b-896491c4dca4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:48.238108 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97eca4e4dbf44f1ea1bace6ab62a7598', 'request_id': 'req-0aef55dc-fb3c-440e-af30-c3001461e132'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:48.244832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97eca4e4dbf44f1ea1bace6ab62a7598', 'request_id': 'req-0aef55dc-fb3c-440e-af30-c3001461e132'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:48.290711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d3372-ab3e-49f1-a34a-f168976b8acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:48.291372 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d3372-ab3e-49f1-a34a-f168976b8acd admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:48.291649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d3372-ab3e-49f1-a34a-f168976b8acd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:48.291920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d3372-ab3e-49f1-a34a-f168976b8acd admin admin] PATH_INFO: `/v3/projects/1f0ef527c804404fbe1a7b63c36f5256/users/1a32cff3e1104e9eb5b0b8adbde79f73/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:48.306463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d3372-ab3e-49f1-a34a-f168976b8acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0ef527c804404fbe1a7b63c36f5256, user_id=1a32cff3e1104e9eb5b0b8adbde79f73, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:48.309090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d3372-ab3e-49f1-a34a-f168976b8acd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:48.335571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] The token cache is being invalidate because project 97eca4e4dbf44f1ea1bace6ab62a7598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:48.336091 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:48.336504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:48.338507 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1286/2609] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:48 2022] DELETE /identity/v3/projects/97eca4e4dbf44f1ea1bace6ab62a7598 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:48.342380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:48.343678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:48.357113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef55dc-fb3c-440e-af30-c3001461e132 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:48.361833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:48.362441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:48.362727 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:48.363005 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] PATH_INFO: `/v3/users/a5a5d8d1e3ec4633bac29cd2d95fafdb` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:48.364204 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5a5d8d1e3ec4633bac29cd2d95fafdb)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:48.366624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:48.416875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1324/2610] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:48 2022] PUT /identity/v3/projects/1f0ef527c804404fbe1a7b63c36f5256/users/1a32cff3e1104e9eb5b0b8adbde79f73/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:48.420863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d3372-ab3e-49f1-a34a-f168976b8acd admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:48.422534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-fb3d3372-ab3e-49f1-a34a-f168976b8acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:48.452667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:48.477293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:48.487974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d3372-ab3e-49f1-a34a-f168976b8acd admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:48.492896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ec4d0-3363-47dc-9404-60a0e03bd964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:48.493529 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ec4d0-3363-47dc-9404-60a0e03bd964 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:48.494075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ec4d0-3363-47dc-9404-60a0e03bd964 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:48.494347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ec4d0-3363-47dc-9404-60a0e03bd964 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:48.494976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ec4d0-3363-47dc-9404-60a0e03bd964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:48.496593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ec4d0-3363-47dc-9404-60a0e03bd964 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:48.507064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1325/2611] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:59:48.511124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ec4d0-3363-47dc-9404-60a0e03bd964 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:48.520437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ec4d0-3363-47dc-9404-60a0e03bd964 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:48.525695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cb3e2-b437-4edb-a93d-192497b5e7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:48.526428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cb3e2-b437-4edb-a93d-192497b5e7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:48.526889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cb3e2-b437-4edb-a93d-192497b5e7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:48.527083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cb3e2-b437-4edb-a93d-192497b5e7c6 admin admin] PATH_INFO: `/v3/projects/1f0ef527c804404fbe1a7b63c36f5256/users/1a32cff3e1104e9eb5b0b8adbde79f73/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:48.528873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cb3e2-b437-4edb-a93d-192497b5e7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0ef527c804404fbe1a7b63c36f5256, user_id=1a32cff3e1104e9eb5b0b8adbde79f73, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:48.531571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cb3e2-b437-4edb-a93d-192497b5e7c6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:48.546358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1326/2612] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:48 2022] PUT /identity/v3/projects/1f0ef527c804404fbe1a7b63c36f5256/users/1a32cff3e1104e9eb5b0b8adbde79f73/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:48.547936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5a5d8d1e3ec4633bac29cd2d95fafdb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:48.548648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5a5d8d1e3ec4633bac29cd2d95fafdb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:48.549363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5a5d8d1e3ec4633bac29cd2d95fafdb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:48.557006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b43c97-9937-4001-b2a2-edf7426de1f7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:48.557315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b43c97-9937-4001-b2a2-edf7426de1f7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:48.557605 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b43c97-9937-4001-b2a2-edf7426de1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:48.562296 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5a5d8d1e3ec4633bac29cd2d95fafdb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:48.569436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5a5d8d1e3ec4633bac29cd2d95fafdb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:48.586642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-15b43c97-9937-4001-b2a2-edf7426de1f7 None None] MFA Rules not processed for user `1a32cff3e1104e9eb5b0b8adbde79f73`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:48.587866 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-15b43c97-9937-4001-b2a2-edf7426de1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:48.655280 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1287/2613] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:48 2022] DELETE /identity/v3/users/a5a5d8d1e3ec4633bac29cd2d95fafdb => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:48.662099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1327/2614] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:48.672675 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:48.674162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:48.744931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2487495-37c0-4701-9b62-27538babae64 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:48.749145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a8d00-332c-4c00-beef-c630fa13dcae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:48.749496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a8d00-332c-4c00-beef-c630fa13dcae service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:48.749773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a8d00-332c-4c00-beef-c630fa13dcae service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:48.750041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a8d00-332c-4c00-beef-c630fa13dcae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:48.811293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a8d00-332c-4c00-beef-c630fa13dcae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:48.812776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a8d00-332c-4c00-beef-c630fa13dcae service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:48.822853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1288/2615] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:59:49.074491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b43c97-9937-4001-b2a2-edf7426de1f7 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:49.076036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-15b43c97-9937-4001-b2a2-edf7426de1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:49.154831 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b43c97-9937-4001-b2a2-edf7426de1f7 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:49.195640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:49.211257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:49.214037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:49.214037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:49.214037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] PATH_INFO: `/v3/projects/39dc52bd3e3a44399d44520a564d6bb9` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:49.214037 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39dc52bd3e3a44399d44520a564d6bb9)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:49.215433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:49.338581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39dc52bd3e3a44399d44520a564d6bb9', 'request_id': 'req-23a0bcc1-9154-4bc6-993e-d29ec35719d4'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:49.344943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39dc52bd3e3a44399d44520a564d6bb9', 'request_id': 'req-23a0bcc1-9154-4bc6-993e-d29ec35719d4'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:49.430273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39dc52bd3e3a44399d44520a564d6bb9', 'request_id': 'req-23a0bcc1-9154-4bc6-993e-d29ec35719d4'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:49.442168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] The token cache is being invalidate because project 39dc52bd3e3a44399d44520a564d6bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:49.442463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:49.443120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-23a0bcc1-9154-4bc6-993e-d29ec35719d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:49.444823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1328/2616] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:49 2022] DELETE /identity/v3/projects/39dc52bd3e3a44399d44520a564d6bb9 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:49.450498 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25418e-85b3-4961-9c56-3b224dd58aed None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:49.450967 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25418e-85b3-4961-9c56-3b224dd58aed None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:49.451062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25418e-85b3-4961-9c56-3b224dd58aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:49.470550 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-3a25418e-85b3-4961-9c56-3b224dd58aed None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:49.471854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3a25418e-85b3-4961-9c56-3b224dd58aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:49.519912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:49.532346 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1289/2617] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:49.540775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ac9e3-a919-41f8-aa23-6bd01eee9bda None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:49.541100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ac9e3-a919-41f8-aa23-6bd01eee9bda None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:49.541496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ac9e3-a919-41f8-aa23-6bd01eee9bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:49.562260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7c8ac9e3-a919-41f8-aa23-6bd01eee9bda None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:49.563459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7c8ac9e3-a919-41f8-aa23-6bd01eee9bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:49.571893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1329/2618] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:49.576355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25418e-85b3-4961-9c56-3b224dd58aed None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:49.586508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25418e-85b3-4961-9c56-3b224dd58aed None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:49.597965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73647d-cb60-47e1-bc08-e14b2d0f0d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:49.598734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73647d-cb60-47e1-bc08-e14b2d0f0d45 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:49.598874 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73647d-cb60-47e1-bc08-e14b2d0f0d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:49.599150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73647d-cb60-47e1-bc08-e14b2d0f0d45 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:49.600348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73647d-cb60-47e1-bc08-e14b2d0f0d45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:59:49.605547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73647d-cb60-47e1-bc08-e14b2d0f0d45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:49.605547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73647d-cb60-47e1-bc08-e14b2d0f0d45 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:49.621322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1290/2619] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:59:49.622416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ac9e3-a919-41f8-aa23-6bd01eee9bda None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:49.633497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ac9e3-a919-41f8-aa23-6bd01eee9bda None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:49.638277 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847ca12-bc69-4214-9ed4-89c50be1f0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:49.638878 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847ca12-bc69-4214-9ed4-89c50be1f0ab admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:49.639197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847ca12-bc69-4214-9ed4-89c50be1f0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:49.639474 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847ca12-bc69-4214-9ed4-89c50be1f0ab admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:49.640122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847ca12-bc69-4214-9ed4-89c50be1f0ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:49.642069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847ca12-bc69-4214-9ed4-89c50be1f0ab admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:49.731590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1330/2620] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:49 2022] POST /identity/v3/projects => generated 409 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:49.736943 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73647d-cb60-47e1-bc08-e14b2d0f0d45 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:49.749354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73647d-cb60-47e1-bc08-e14b2d0f0d45 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:49.809896 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba629c4b-d7b7-4d6d-be1f-612927d3ac8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:49.810542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629c4b-d7b7-4d6d-be1f-612927d3ac8e admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:49.810821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629c4b-d7b7-4d6d-be1f-612927d3ac8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:49.811093 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba629c4b-d7b7-4d6d-be1f-612927d3ac8e admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:49.811720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba629c4b-d7b7-4d6d-be1f-612927d3ac8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:49.813678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba629c4b-d7b7-4d6d-be1f-612927d3ac8e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:49.930568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1291/2621] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:49 2022] POST /identity/v3/users => generated 305 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:49.933709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847ca12-bc69-4214-9ed4-89c50be1f0ab admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:49.946639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847ca12-bc69-4214-9ed4-89c50be1f0ab admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:49.951691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936aa467-5c59-4f2d-ae9e-ce6f8e8541e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:49.952224 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aa467-5c59-4f2d-ae9e-ce6f8e8541e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:49.955658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aa467-5c59-4f2d-ae9e-ce6f8e8541e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:49.955658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936aa467-5c59-4f2d-ae9e-ce6f8e8541e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:49.955658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936aa467-5c59-4f2d-ae9e-ce6f8e8541e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:49.955658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936aa467-5c59-4f2d-ae9e-ce6f8e8541e3 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:49.966072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1331/2622] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:49.970208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba629c4b-d7b7-4d6d-be1f-612927d3ac8e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:49.981635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba629c4b-d7b7-4d6d-be1f-612927d3ac8e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:49.986442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4ea01-e158-4444-b679-ac28e2b3d9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:49.987113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ea01-e158-4444-b679-ac28e2b3d9df admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:49.987396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ea01-e158-4444-b679-ac28e2b3d9df admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:49.990167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ea01-e158-4444-b679-ac28e2b3d9df admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:49.990167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4ea01-e158-4444-b679-ac28e2b3d9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:49.990167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4ea01-e158-4444-b679-ac28e2b3d9df admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:50.000221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1292/2623] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:50.004351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936aa467-5c59-4f2d-ae9e-ce6f8e8541e3 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:50.014610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936aa467-5c59-4f2d-ae9e-ce6f8e8541e3 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:50.019209 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aaaa32-bf7f-4191-9fad-7ee95c88af8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:50.019853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aaaa32-bf7f-4191-9fad-7ee95c88af8d admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.020132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aaaa32-bf7f-4191-9fad-7ee95c88af8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.020405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aaaa32-bf7f-4191-9fad-7ee95c88af8d admin admin] PATH_INFO: `/v3/projects/ac942a71f774453ab592e59618bf20a1/users/82dbaa4c11e348a3bc029693410adacb/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.035621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aaaa32-bf7f-4191-9fad-7ee95c88af8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac942a71f774453ab592e59618bf20a1, user_id=82dbaa4c11e348a3bc029693410adacb, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:50.040438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aaaa32-bf7f-4191-9fad-7ee95c88af8d admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:50.066069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4ea01-e158-4444-b679-ac28e2b3d9df admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:50.067788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1ec4ea01-e158-4444-b679-ac28e2b3d9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:50.080630 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4ea01-e158-4444-b679-ac28e2b3d9df admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:50.085944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:50.086565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.086847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.087121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] PATH_INFO: `/v3/users/d4b0c5a55bcc40aabec5c9db8cbd31e0` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.088285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4b0c5a55bcc40aabec5c9db8cbd31e0)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:50.090263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:50.130252 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1332/2624] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:50 2022] PUT /identity/v3/projects/ac942a71f774453ab592e59618bf20a1/users/82dbaa4c11e348a3bc029693410adacb/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:50.141062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2dea5-ebb9-41d6-a7d7-78b0633578d1 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.141363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2dea5-ebb9-41d6-a7d7-78b0633578d1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.141670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2dea5-ebb9-41d6-a7d7-78b0633578d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.170426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1af2dea5-ebb9-41d6-a7d7-78b0633578d1 None None] MFA Rules not processed for user `82dbaa4c11e348a3bc029693410adacb`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:50.171619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1af2dea5-ebb9-41d6-a7d7-78b0633578d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:50.232726 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:50.237950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1333/2625] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:50 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:59:50.243113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2dea5-ebb9-41d6-a7d7-78b0633578d1 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:50.292959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4b0c5a55bcc40aabec5c9db8cbd31e0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:50.293651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4b0c5a55bcc40aabec5c9db8cbd31e0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:50.294265 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4b0c5a55bcc40aabec5c9db8cbd31e0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:50.303902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2dea5-ebb9-41d6-a7d7-78b0633578d1 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:50.307231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4b0c5a55bcc40aabec5c9db8cbd31e0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:50.310383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326c3f6-83d2-476e-b9ec-f0bf3941374b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:50.310383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326c3f6-83d2-476e-b9ec-f0bf3941374b admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.310383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326c3f6-83d2-476e-b9ec-f0bf3941374b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.310383 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326c3f6-83d2-476e-b9ec-f0bf3941374b admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.310794 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326c3f6-83d2-476e-b9ec-f0bf3941374b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:50.312517 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326c3f6-83d2-476e-b9ec-f0bf3941374b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:50.314467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-075b011e-0b3d-411c-97d2-ee7b61c168ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4b0c5a55bcc40aabec5c9db8cbd31e0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:50.446778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1293/2626] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:50 2022] DELETE /identity/v3/users/d4b0c5a55bcc40aabec5c9db8cbd31e0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:50.451155 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1334/2627] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:50 2022] POST /identity/v3/projects => generated 405 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:59:50.451318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a05fb-0786-4da7-b855-81cd1c5d3357 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.451746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a05fb-0786-4da7-b855-81cd1c5d3357 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.451746 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a05fb-0786-4da7-b855-81cd1c5d3357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.452140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326c3f6-83d2-476e-b9ec-f0bf3941374b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:50.453885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b326c3f6-83d2-476e-b9ec-f0bf3941374b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:50.471704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-949a05fb-0786-4da7-b855-81cd1c5d3357 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:50.473743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-949a05fb-0786-4da7-b855-81cd1c5d3357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:50.500070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:50.520532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326c3f6-83d2-476e-b9ec-f0bf3941374b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:50.525853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18c4bd-88fb-4bbc-9a95-3d90730a717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:50.526460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18c4bd-88fb-4bbc-9a95-3d90730a717e admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.526771 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18c4bd-88fb-4bbc-9a95-3d90730a717e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.527048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18c4bd-88fb-4bbc-9a95-3d90730a717e admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.527677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18c4bd-88fb-4bbc-9a95-3d90730a717e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:50.529622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18c4bd-88fb-4bbc-9a95-3d90730a717e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:50.533548 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1294/2628] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:50.544288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a05fb-0786-4da7-b855-81cd1c5d3357 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:50.545677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-949a05fb-0786-4da7-b855-81cd1c5d3357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:50.603954 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a05fb-0786-4da7-b855-81cd1c5d3357 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:50.611225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dfff9-a030-4ddc-b1de-106e7bb8e873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:50.611225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dfff9-a030-4ddc-b1de-106e7bb8e873 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.611225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dfff9-a030-4ddc-b1de-106e7bb8e873 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.611225 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dfff9-a030-4ddc-b1de-106e7bb8e873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.619313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dfff9-a030-4ddc-b1de-106e7bb8e873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:50.620748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dfff9-a030-4ddc-b1de-106e7bb8e873 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:50.630313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1295/2629] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:59:50.665530 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1335/2630] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:50 2022] POST /identity/v3/users => generated 309 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:50.670067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dfff9-a030-4ddc-b1de-106e7bb8e873 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:50.684653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dfff9-a030-4ddc-b1de-106e7bb8e873 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:50.691501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c6221-e3dc-4702-b774-485a812d506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:50.692502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c6221-e3dc-4702-b774-485a812d506a admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.692979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c6221-e3dc-4702-b774-485a812d506a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.693494 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c6221-e3dc-4702-b774-485a812d506a admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.694593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c6221-e3dc-4702-b774-485a812d506a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:50.697208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c6221-e3dc-4702-b774-485a812d506a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:50.708641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1296/2631] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:50.715405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18c4bd-88fb-4bbc-9a95-3d90730a717e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:50.727384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18c4bd-88fb-4bbc-9a95-3d90730a717e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:50.732303 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d53f0d-da5e-4c39-9d15-e961e9be36c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:50.732912 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d53f0d-da5e-4c39-9d15-e961e9be36c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.733238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d53f0d-da5e-4c39-9d15-e961e9be36c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.733518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d53f0d-da5e-4c39-9d15-e961e9be36c2 admin admin] PATH_INFO: `/v3/projects/08c36b568df647cfb667db158b356c3c/users/a359ac4d7a94463dbf09c82f92c1f164/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.749776 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d53f0d-da5e-4c39-9d15-e961e9be36c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c36b568df647cfb667db158b356c3c, user_id=a359ac4d7a94463dbf09c82f92c1f164, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:50.752449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d53f0d-da5e-4c39-9d15-e961e9be36c2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:50.815478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1336/2632] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:50 2022] PUT /identity/v3/projects/08c36b568df647cfb667db158b356c3c/users/a359ac4d7a94463dbf09c82f92c1f164/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:50.820116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c6221-e3dc-4702-b774-485a812d506a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:50.834641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c6221-e3dc-4702-b774-485a812d506a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:50.895201 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309ba14-6e40-4c71-81f7-8d8a5a3282d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:50.895810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309ba14-6e40-4c71-81f7-8d8a5a3282d8 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.896089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309ba14-6e40-4c71-81f7-8d8a5a3282d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.896545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309ba14-6e40-4c71-81f7-8d8a5a3282d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.897299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309ba14-6e40-4c71-81f7-8d8a5a3282d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:50.898928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309ba14-6e40-4c71-81f7-8d8a5a3282d8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:50.909719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1297/2633] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:50.914313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d53f0d-da5e-4c39-9d15-e961e9be36c2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:50.924788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d53f0d-da5e-4c39-9d15-e961e9be36c2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:50.929938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d6569-7cbd-4400-9ecf-3e9d848273eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:50.930588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d6569-7cbd-4400-9ecf-3e9d848273eb admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.930867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d6569-7cbd-4400-9ecf-3e9d848273eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.931144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d6569-7cbd-4400-9ecf-3e9d848273eb admin admin] PATH_INFO: `/v3/projects/08c36b568df647cfb667db158b356c3c/users/a359ac4d7a94463dbf09c82f92c1f164/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.933088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d6569-7cbd-4400-9ecf-3e9d848273eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c36b568df647cfb667db158b356c3c, user_id=a359ac4d7a94463dbf09c82f92c1f164, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:50.935690 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d6569-7cbd-4400-9ecf-3e9d848273eb admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:50.946962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1337/2634] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:50 2022] PUT /identity/v3/projects/08c36b568df647cfb667db158b356c3c/users/a359ac4d7a94463dbf09c82f92c1f164/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:50.960685 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1c19-d868-40f9-a95e-ce331d46b244 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:50.961026 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1c19-d868-40f9-a95e-ce331d46b244 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:50.961336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1c19-d868-40f9-a95e-ce331d46b244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:50.990955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-981d1c19-d868-40f9-a95e-ce331d46b244 None None] MFA Rules not processed for user `a359ac4d7a94463dbf09c82f92c1f164`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:50.992163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-981d1c19-d868-40f9-a95e-ce331d46b244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:51.034217 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d6569-7cbd-4400-9ecf-3e9d848273eb admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:51.036413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ae7d6569-7cbd-4400-9ecf-3e9d848273eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:51.043745 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=100 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:51.079347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1298/2635] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:50 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:51.082896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:51.091793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d1c19-d868-40f9-a95e-ce331d46b244 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:51.096333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-981d1c19-d868-40f9-a95e-ce331d46b244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:51.109566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d6569-7cbd-4400-9ecf-3e9d848273eb admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:51.114239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:51.114872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:51.115716 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:51.116002 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] PATH_INFO: `/v3/projects/588c9c39bd694f1e860679848d754b3c` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:51.117158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=588c9c39bd694f1e860679848d754b3c)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:51.119369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:51.176808 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d1c19-d868-40f9-a95e-ce331d46b244 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:51.181351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616067d8-d29d-436a-8bd2-0c54d0c2a56d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:51.181959 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616067d8-d29d-436a-8bd2-0c54d0c2a56d service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:51.182275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616067d8-d29d-436a-8bd2-0c54d0c2a56d service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:51.182524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616067d8-d29d-436a-8bd2-0c54d0c2a56d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:51.191742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616067d8-d29d-436a-8bd2-0c54d0c2a56d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:51.193232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616067d8-d29d-436a-8bd2-0c54d0c2a56d service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:51.202702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1299/2636] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:51 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:59:51.228145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '588c9c39bd694f1e860679848d754b3c', 'request_id': 'req-597b2571-3947-4bb5-be54-458278489160'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:51.234960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '588c9c39bd694f1e860679848d754b3c', 'request_id': 'req-597b2571-3947-4bb5-be54-458278489160'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:51.312610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '588c9c39bd694f1e860679848d754b3c', 'request_id': 'req-597b2571-3947-4bb5-be54-458278489160'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:51.323855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] The token cache is being invalidate because project 588c9c39bd694f1e860679848d754b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:51.324129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:51.324772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:51.326369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1338/2637] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:51 2022] DELETE /identity/v3/projects/588c9c39bd694f1e860679848d754b3c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 18:59:51.330680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616067d8-d29d-436a-8bd2-0c54d0c2a56d service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:51.341792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616067d8-d29d-436a-8bd2-0c54d0c2a56d service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:51.399796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:51.400425 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:51.400702 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:51.400973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] PATH_INFO: `/v3/users/1a32cff3e1104e9eb5b0b8adbde79f73` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:51.402125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a32cff3e1104e9eb5b0b8adbde79f73)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:51.404036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:51.617485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a32cff3e1104e9eb5b0b8adbde79f73'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:51.618163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a32cff3e1104e9eb5b0b8adbde79f73'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:51.619045 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a32cff3e1104e9eb5b0b8adbde79f73'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:51.636421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a32cff3e1104e9eb5b0b8adbde79f73'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:51.641257 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:51.645777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:51.645777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:51.646014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a32cff3e1104e9eb5b0b8adbde79f73'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:51.701293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b2571-3947-4bb5-be54-458278489160 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:51.704894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6f652-dd8b-4717-979b-e3a136fca111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:51.705525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6f652-dd8b-4717-979b-e3a136fca111 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:51.705804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6f652-dd8b-4717-979b-e3a136fca111 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:51.706073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6f652-dd8b-4717-979b-e3a136fca111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:51.707813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-80f6f652-dd8b-4717-979b-e3a136fca111 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:51.750718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1300/2638] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:51 2022] DELETE /identity/v3/users/1a32cff3e1104e9eb5b0b8adbde79f73 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:51.768130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6f652-dd8b-4717-979b-e3a136fca111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:51.773206 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6f652-dd8b-4717-979b-e3a136fca111 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:51.779390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1339/2639] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:51 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 18:59:52.145628 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:52.147310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:52.219887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea84fe-1281-4c87-85bb-535bf2b96860 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:52.225073 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:52.225711 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:52.226023 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:52.226358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] PATH_INFO: `/v3/projects/1f0ef527c804404fbe1a7b63c36f5256` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:52.227578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f0ef527c804404fbe1a7b63c36f5256)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:52.229575 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:52.342149 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f0ef527c804404fbe1a7b63c36f5256', 'request_id': 'req-320a5081-0ecf-4f00-9087-d8820f250dd9'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:52.348551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f0ef527c804404fbe1a7b63c36f5256', 'request_id': 'req-320a5081-0ecf-4f00-9087-d8820f250dd9'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:52.354903 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f0ef527c804404fbe1a7b63c36f5256', 'request_id': 'req-320a5081-0ecf-4f00-9087-d8820f250dd9'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:52.418592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] The token cache is being invalidate because project 1f0ef527c804404fbe1a7b63c36f5256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:52.418832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:52.419480 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-320a5081-0ecf-4f00-9087-d8820f250dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:52.421203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1301/2640] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:52 2022] DELETE /identity/v3/projects/1f0ef527c804404fbe1a7b63c36f5256 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:52.431106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25422e4-949f-490e-8c95-50fffe2ca13c None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:52.431545 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25422e4-949f-490e-8c95-50fffe2ca13c None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:52.431639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25422e4-949f-490e-8c95-50fffe2ca13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:52.452971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f25422e4-949f-490e-8c95-50fffe2ca13c None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:52.454281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f25422e4-949f-490e-8c95-50fffe2ca13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:52.468450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:52.518185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1340/2641] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:52.527036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d93173-44b9-44d7-9d9c-64e2ac7b68d9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:52.527173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d93173-44b9-44d7-9d9c-64e2ac7b68d9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:52.527601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d93173-44b9-44d7-9d9c-64e2ac7b68d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:52.548904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a9d93173-44b9-44d7-9d9c-64e2ac7b68d9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:52.550214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a9d93173-44b9-44d7-9d9c-64e2ac7b68d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:52.558302 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1302/2642] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:52.562567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25422e4-949f-490e-8c95-50fffe2ca13c None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:52.573070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25422e4-949f-490e-8c95-50fffe2ca13c None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:52.577682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f668a-8911-40e8-95c4-42036db3e4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:52.578283 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f668a-8911-40e8-95c4-42036db3e4ba admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:52.578565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f668a-8911-40e8-95c4-42036db3e4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:52.578842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f668a-8911-40e8-95c4-42036db3e4ba admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:52.579486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f668a-8911-40e8-95c4-42036db3e4ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:59:52.579792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f668a-8911-40e8-95c4-42036db3e4ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:52.581440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f668a-8911-40e8-95c4-42036db3e4ba admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:52.594513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1341/2643] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:59:52.598877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d93173-44b9-44d7-9d9c-64e2ac7b68d9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:52.608766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d93173-44b9-44d7-9d9c-64e2ac7b68d9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:52.613549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ca2b7-fc2d-404c-a9b6-02345f5850a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:52.614150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ca2b7-fc2d-404c-a9b6-02345f5850a4 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:52.614434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ca2b7-fc2d-404c-a9b6-02345f5850a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:52.614709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ca2b7-fc2d-404c-a9b6-02345f5850a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:52.615375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ca2b7-fc2d-404c-a9b6-02345f5850a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:52.617462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ca2b7-fc2d-404c-a9b6-02345f5850a4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:52.734091 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1303/2644] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:52 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 18:59:52.738236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f668a-8911-40e8-95c4-42036db3e4ba admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:52.749214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f668a-8911-40e8-95c4-42036db3e4ba admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:52.808134 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d3f1d-2f07-4c95-be49-9e7d54f1fa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:52.808741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d3f1d-2f07-4c95-be49-9e7d54f1fa77 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:52.809022 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d3f1d-2f07-4c95-be49-9e7d54f1fa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:52.809342 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d3f1d-2f07-4c95-be49-9e7d54f1fa77 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:52.810010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d3f1d-2f07-4c95-be49-9e7d54f1fa77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:52.811927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d3f1d-2f07-4c95-be49-9e7d54f1fa77 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:52.903600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1342/2645] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:52 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:52.907765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ca2b7-fc2d-404c-a9b6-02345f5850a4 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:52.918832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ca2b7-fc2d-404c-a9b6-02345f5850a4 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:52.923732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ccaf7-181d-4ea4-b214-dfb4708717f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:52.924327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ccaf7-181d-4ea4-b214-dfb4708717f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:52.924622 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ccaf7-181d-4ea4-b214-dfb4708717f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:52.924880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ccaf7-181d-4ea4-b214-dfb4708717f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:52.925549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ccaf7-181d-4ea4-b214-dfb4708717f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:52.927130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ccaf7-181d-4ea4-b214-dfb4708717f5 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:52.937840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1304/2646] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:52.941908 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d3f1d-2f07-4c95-be49-9e7d54f1fa77 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:52.952660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d3f1d-2f07-4c95-be49-9e7d54f1fa77 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:52.957438 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73822fbb-9eed-4935-bdcf-8099f173e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:52.958034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73822fbb-9eed-4935-bdcf-8099f173e38c admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:52.958313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73822fbb-9eed-4935-bdcf-8099f173e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:52.958585 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73822fbb-9eed-4935-bdcf-8099f173e38c admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:52.959379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73822fbb-9eed-4935-bdcf-8099f173e38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:52.961007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73822fbb-9eed-4935-bdcf-8099f173e38c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:52.971671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1343/2647] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:52.976061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ccaf7-181d-4ea4-b214-dfb4708717f5 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:52.986346 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ccaf7-181d-4ea4-b214-dfb4708717f5 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:52.991668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ad881-261d-4de8-a9c7-b8b371d89c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:52.992351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ad881-261d-4de8-a9c7-b8b371d89c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:52.992626 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ad881-261d-4de8-a9c7-b8b371d89c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:52.992930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ad881-261d-4de8-a9c7-b8b371d89c60 admin admin] PATH_INFO: `/v3/projects/8ff60b064b3a487db8f6154f91a57f5a/users/4608d8666dcd438c8e29c2b38819814b/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:53.010086 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ad881-261d-4de8-a9c7-b8b371d89c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff60b064b3a487db8f6154f91a57f5a, user_id=4608d8666dcd438c8e29c2b38819814b, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:53.054658 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ad881-261d-4de8-a9c7-b8b371d89c60 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:53.272815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1305/2648] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:52 2022] PUT /identity/v3/projects/8ff60b064b3a487db8f6154f91a57f5a/users/4608d8666dcd438c8e29c2b38819814b/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:53.283300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68a720-eda3-45db-9623-989b6c73dadb None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:53.283592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68a720-eda3-45db-9623-989b6c73dadb None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:53.283874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68a720-eda3-45db-9623-989b6c73dadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:53.312938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-3f68a720-eda3-45db-9623-989b6c73dadb None None] MFA Rules not processed for user `4608d8666dcd438c8e29c2b38819814b`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:53.314179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3f68a720-eda3-45db-9623-989b6c73dadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:53.382654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1344/2649] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:53.387301 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ad881-261d-4de8-a9c7-b8b371d89c60 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:53.401171 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:53.450155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ad881-261d-4de8-a9c7-b8b371d89c60 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:53.454751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05ca00-6312-4d3c-a396-d015d3bf8eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:53.455341 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05ca00-6312-4d3c-a396-d015d3bf8eba admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:53.455623 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05ca00-6312-4d3c-a396-d015d3bf8eba admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:53.455895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05ca00-6312-4d3c-a396-d015d3bf8eba admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:53.456523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05ca00-6312-4d3c-a396-d015d3bf8eba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:53.458433 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05ca00-6312-4d3c-a396-d015d3bf8eba admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:53.589983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1306/2650] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:53 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:53.594207 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68a720-eda3-45db-9623-989b6c73dadb None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:53.606765 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68a720-eda3-45db-9623-989b6c73dadb None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:53.621799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:53.668266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdf4e4-f35b-4123-95f5-929441cfa3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:53.668957 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdf4e4-f35b-4123-95f5-929441cfa3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:53.669326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdf4e4-f35b-4123-95f5-929441cfa3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:53.669596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdf4e4-f35b-4123-95f5-929441cfa3e8 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:53.670248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdf4e4-f35b-4123-95f5-929441cfa3e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:53.672326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdf4e4-f35b-4123-95f5-929441cfa3e8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:53.776407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1345/2651] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:53 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:53.780591 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05ca00-6312-4d3c-a396-d015d3bf8eba admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:53.792124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05ca00-6312-4d3c-a396-d015d3bf8eba admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:53.796971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c591c0-f6e6-4ccd-9c5d-1ff895a71051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:53.797606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c591c0-f6e6-4ccd-9c5d-1ff895a71051 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:53.798025 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c591c0-f6e6-4ccd-9c5d-1ff895a71051 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:53.798162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c591c0-f6e6-4ccd-9c5d-1ff895a71051 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:53.798797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c591c0-f6e6-4ccd-9c5d-1ff895a71051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:53.800477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c591c0-f6e6-4ccd-9c5d-1ff895a71051 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:53.811522 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1307/2652] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:53.815566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdf4e4-f35b-4123-95f5-929441cfa3e8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:53.825789 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdf4e4-f35b-4123-95f5-929441cfa3e8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:53.830442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc07c03-d995-4f3a-a53d-8dada3274af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:53.831110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07c03-d995-4f3a-a53d-8dada3274af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:53.831395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07c03-d995-4f3a-a53d-8dada3274af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:53.831671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc07c03-d995-4f3a-a53d-8dada3274af4 admin admin] PATH_INFO: `/v3/projects/384fa57f18ac455c84388e5baddc6b51/users/3e561e497d274734a340d5751c5daadb/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:53.847480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc07c03-d995-4f3a-a53d-8dada3274af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=384fa57f18ac455c84388e5baddc6b51, user_id=3e561e497d274734a340d5751c5daadb, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:53.851093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc07c03-d995-4f3a-a53d-8dada3274af4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:53.932010 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1346/2653] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:53 2022] PUT /identity/v3/projects/384fa57f18ac455c84388e5baddc6b51/users/3e561e497d274734a340d5751c5daadb/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:53.935940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c591c0-f6e6-4ccd-9c5d-1ff895a71051 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:53.946537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c591c0-f6e6-4ccd-9c5d-1ff895a71051 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:54.011883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cc229-f7dd-4bde-b3cf-889ea2f4742d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:54.012501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc229-f7dd-4bde-b3cf-889ea2f4742d admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:54.012783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc229-f7dd-4bde-b3cf-889ea2f4742d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:54.013085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc229-f7dd-4bde-b3cf-889ea2f4742d admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:54.013835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cc229-f7dd-4bde-b3cf-889ea2f4742d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:54.015413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cc229-f7dd-4bde-b3cf-889ea2f4742d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:54.026372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1308/2654] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:54.030565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc07c03-d995-4f3a-a53d-8dada3274af4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:54.041292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc07c03-d995-4f3a-a53d-8dada3274af4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:54.046835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28365f-9ec7-4b51-86b4-27f67df789ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:54.050519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28365f-9ec7-4b51-86b4-27f67df789ba admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:54.050519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28365f-9ec7-4b51-86b4-27f67df789ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:54.050519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28365f-9ec7-4b51-86b4-27f67df789ba admin admin] PATH_INFO: `/v3/projects/384fa57f18ac455c84388e5baddc6b51/users/3e561e497d274734a340d5751c5daadb/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:54.050519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28365f-9ec7-4b51-86b4-27f67df789ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=384fa57f18ac455c84388e5baddc6b51, user_id=3e561e497d274734a340d5751c5daadb, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:54.052711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28365f-9ec7-4b51-86b4-27f67df789ba admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:54.063039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1347/2655] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:54 2022] PUT /identity/v3/projects/384fa57f18ac455c84388e5baddc6b51/users/3e561e497d274734a340d5751c5daadb/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:54.129901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5609ae-677a-4fb0-ba3d-b5904ee73f8d None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:54.130191 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5609ae-677a-4fb0-ba3d-b5904ee73f8d None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:54.130472 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5609ae-677a-4fb0-ba3d-b5904ee73f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:54.159651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-bf5609ae-677a-4fb0-ba3d-b5904ee73f8d None None] MFA Rules not processed for user `3e561e497d274734a340d5751c5daadb`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:54.160854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bf5609ae-677a-4fb0-ba3d-b5904ee73f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:54.229677 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1309/2656] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:54.240688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28365f-9ec7-4b51-86b4-27f67df789ba admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:54.242375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5c28365f-9ec7-4b51-86b4-27f67df789ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:54.299972 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28365f-9ec7-4b51-86b4-27f67df789ba admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:54.304590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195420c-2fec-42b6-bf75-90a61b929888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:54.305223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195420c-2fec-42b6-bf75-90a61b929888 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:54.305512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195420c-2fec-42b6-bf75-90a61b929888 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:54.305790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195420c-2fec-42b6-bf75-90a61b929888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:54.314680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195420c-2fec-42b6-bf75-90a61b929888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:54.316090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195420c-2fec-42b6-bf75-90a61b929888 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:54.326568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1348/2657] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:59:54.887413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5609ae-677a-4fb0-ba3d-b5904ee73f8d None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:54.897154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:54.949502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5609ae-677a-4fb0-ba3d-b5904ee73f8d None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:54.954000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:54.954620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:54.954895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:54.955168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] PATH_INFO: `/v3/users/4608d8666dcd438c8e29c2b38819814b` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:54.956263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4608d8666dcd438c8e29c2b38819814b)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:54.958240 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:55.117684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4608d8666dcd438c8e29c2b38819814b'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:55.118274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4608d8666dcd438c8e29c2b38819814b'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:55.119126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4608d8666dcd438c8e29c2b38819814b'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:55.132041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4608d8666dcd438c8e29c2b38819814b'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:55.138951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4608d8666dcd438c8e29c2b38819814b'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:55.229585 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1310/2658] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:54 2022] DELETE /identity/v3/users/4608d8666dcd438c8e29c2b38819814b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:55.234233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3986ec-a5af-4887-bb5a-09d25210cad7 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:55.234706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3986ec-a5af-4887-bb5a-09d25210cad7 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:55.234826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3986ec-a5af-4887-bb5a-09d25210cad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:55.254362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-9e3986ec-a5af-4887-bb5a-09d25210cad7 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:55.256174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9e3986ec-a5af-4887-bb5a-09d25210cad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:55.314189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1349/2659] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 18:59:55.324597 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:55.326074 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:55.384321 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89e818-b901-4865-a1fb-073cc178a70c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:55.387823 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d518ee8-1530-492b-bf46-854fee53a6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:55.388418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d518ee8-1530-492b-bf46-854fee53a6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:55.388695 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d518ee8-1530-492b-bf46-854fee53a6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:55.388965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d518ee8-1530-492b-bf46-854fee53a6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:55.398640 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d518ee8-1530-492b-bf46-854fee53a6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:55.400107 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d518ee8-1530-492b-bf46-854fee53a6c8 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:55.409516 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1311/2660] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:59:55.806060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3986ec-a5af-4887-bb5a-09d25210cad7 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:55.807489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9e3986ec-a5af-4887-bb5a-09d25210cad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:55.821687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3986ec-a5af-4887-bb5a-09d25210cad7 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:55.826396 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:55.827061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:55.827341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:55.827609 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] PATH_INFO: `/v3/projects/8ff60b064b3a487db8f6154f91a57f5a` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:55.828718 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff60b064b3a487db8f6154f91a57f5a)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:55.830973 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:55.838191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:55.935459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff60b064b3a487db8f6154f91a57f5a', 'request_id': 'req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:55.941597 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff60b064b3a487db8f6154f91a57f5a', 'request_id': 'req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.003542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff60b064b3a487db8f6154f91a57f5a', 'request_id': 'req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.014944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] The token cache is being invalidate because project 8ff60b064b3a487db8f6154f91a57f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:56.019858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.019858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.019858 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1350/2661] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:55 2022] DELETE /identity/v3/projects/8ff60b064b3a487db8f6154f91a57f5a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:56.021496 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d518ee8-1530-492b-bf46-854fee53a6c8 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:56.032359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d518ee8-1530-492b-bf46-854fee53a6c8 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:56.062070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:56.089537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:56.090167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:56.090450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:56.090721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] PATH_INFO: `/v3/users/3e561e497d274734a340d5751c5daadb` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:56.091826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e561e497d274734a340d5751c5daadb)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:56.093895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:56.260440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e561e497d274734a340d5751c5daadb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.261115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e561e497d274734a340d5751c5daadb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.261718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e561e497d274734a340d5751c5daadb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.274885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e561e497d274734a340d5751c5daadb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.281799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3746dc13-13ad-4337-b25e-d4a95a6139eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e561e497d274734a340d5751c5daadb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.355059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1312/2662] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 18:59:56 2022] DELETE /identity/v3/users/3e561e497d274734a340d5751c5daadb => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:56.731461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:56.732941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:56.746420 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6511c-1fa4-4f30-ba33-3fd74234ab6f admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:56.805003 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:56.805693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:56.806000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:56.806273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] PATH_INFO: `/v3/projects/384fa57f18ac455c84388e5baddc6b51` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:56.807375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=384fa57f18ac455c84388e5baddc6b51)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:56.809638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:56.919233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384fa57f18ac455c84388e5baddc6b51', 'request_id': 'req-d4d5589f-a378-44af-a069-500459621d15'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.926019 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384fa57f18ac455c84388e5baddc6b51', 'request_id': 'req-d4d5589f-a378-44af-a069-500459621d15'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.983644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384fa57f18ac455c84388e5baddc6b51', 'request_id': 'req-d4d5589f-a378-44af-a069-500459621d15'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.995954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] The token cache is being invalidate because project 384fa57f18ac455c84388e5baddc6b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 18:59:56.996332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.996960 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d4d5589f-a378-44af-a069-500459621d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 18:59:56.998471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1351/2663] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 18:59:56 2022] DELETE /identity/v3/projects/384fa57f18ac455c84388e5baddc6b51 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:57.003424 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80e16b-bbaf-4a94-ba78-1c8d655904a7 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:57.003716 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80e16b-bbaf-4a94-ba78-1c8d655904a7 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:57.003996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80e16b-bbaf-4a94-ba78-1c8d655904a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:57.023633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-4e80e16b-bbaf-4a94-ba78-1c8d655904a7 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:57.024845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-4e80e16b-bbaf-4a94-ba78-1c8d655904a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:57.088217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1313/2664] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:57.101820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d10c-2f99-48a3-a93f-a699e05bb2df None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:57.102110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d10c-2f99-48a3-a93f-a699e05bb2df None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:57.102685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d10c-2f99-48a3-a93f-a699e05bb2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:57.123598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-7702d10c-2f99-48a3-a93f-a699e05bb2df None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:57.124818 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7702d10c-2f99-48a3-a93f-a699e05bb2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:57.133762 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1352/2665] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:57.138911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80e16b-bbaf-4a94-ba78-1c8d655904a7 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:57.148627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80e16b-bbaf-4a94-ba78-1c8d655904a7 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:57.153755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce75f90-9a5e-4a11-acae-aab21726f70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:57.154358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce75f90-9a5e-4a11-acae-aab21726f70b admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:57.154635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce75f90-9a5e-4a11-acae-aab21726f70b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:57.154906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce75f90-9a5e-4a11-acae-aab21726f70b admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:57.155551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce75f90-9a5e-4a11-acae-aab21726f70b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 18:59:57.155852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce75f90-9a5e-4a11-acae-aab21726f70b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:57.157542 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce75f90-9a5e-4a11-acae-aab21726f70b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:57.169802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1314/2666] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 18:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 18:59:57.177484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d10c-2f99-48a3-a93f-a699e05bb2df None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:57.188740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d10c-2f99-48a3-a93f-a699e05bb2df None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:57.193896 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed39854-6f8a-4163-9f66-8ca6b834a7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:57.194606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed39854-6f8a-4163-9f66-8ca6b834a7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:57.194881 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed39854-6f8a-4163-9f66-8ca6b834a7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:57.195147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed39854-6f8a-4163-9f66-8ca6b834a7d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:57.195775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed39854-6f8a-4163-9f66-8ca6b834a7d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:57.197830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed39854-6f8a-4163-9f66-8ca6b834a7d6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:57.213963 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:57.299308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1353/2667] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 18:59:57 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:57.303627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce75f90-9a5e-4a11-acae-aab21726f70b admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:57.313660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce75f90-9a5e-4a11-acae-aab21726f70b admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:57.355881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 18:59:57.380816 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e39c02-0ebd-4321-9376-81c8a7b0550e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:57.381495 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e39c02-0ebd-4321-9376-81c8a7b0550e admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:57.381827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e39c02-0ebd-4321-9376-81c8a7b0550e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:57.382099 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e39c02-0ebd-4321-9376-81c8a7b0550e admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:57.382732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e39c02-0ebd-4321-9376-81c8a7b0550e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:57.384660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e39c02-0ebd-4321-9376-81c8a7b0550e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:57.467951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1315/2668] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 18:59:57 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 18:59:57.472144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed39854-6f8a-4163-9f66-8ca6b834a7d6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:57.483125 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed39854-6f8a-4163-9f66-8ca6b834a7d6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:57.487720 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08819d-b5a9-4702-9b0e-57dbab3eca6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:57.488377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08819d-b5a9-4702-9b0e-57dbab3eca6e admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:57.488658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08819d-b5a9-4702-9b0e-57dbab3eca6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:57.488931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08819d-b5a9-4702-9b0e-57dbab3eca6e admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:57.489595 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08819d-b5a9-4702-9b0e-57dbab3eca6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:57.491195 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08819d-b5a9-4702-9b0e-57dbab3eca6e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:57.501844 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1354/2669] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 18:59:57.505879 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e39c02-0ebd-4321-9376-81c8a7b0550e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:57.515833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e39c02-0ebd-4321-9376-81c8a7b0550e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:57.520478 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f060d3e-93b1-4e92-9907-bb29b6668c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:57.521105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f060d3e-93b1-4e92-9907-bb29b6668c39 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:57.521392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f060d3e-93b1-4e92-9907-bb29b6668c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:57.521667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f060d3e-93b1-4e92-9907-bb29b6668c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:57.522843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f060d3e-93b1-4e92-9907-bb29b6668c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:57.524558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f060d3e-93b1-4e92-9907-bb29b6668c39 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:57.534670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1316/2670] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 18:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 18:59:57.539361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08819d-b5a9-4702-9b0e-57dbab3eca6e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:57.549412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08819d-b5a9-4702-9b0e-57dbab3eca6e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:57.554047 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55effc-d6ec-4800-b76e-0131e7edffd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:57.554705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55effc-d6ec-4800-b76e-0131e7edffd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:57.554985 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55effc-d6ec-4800-b76e-0131e7edffd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:57.555262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55effc-d6ec-4800-b76e-0131e7edffd4 admin admin] PATH_INFO: `/v3/projects/1235f57770f34c87aeb3369fa8e04b54/users/1a5c2d856c8f4bfc9d59b976ca4371ee/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:57.570080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55effc-d6ec-4800-b76e-0131e7edffd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1235f57770f34c87aeb3369fa8e04b54, user_id=1a5c2d856c8f4bfc9d59b976ca4371ee, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:57.572736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55effc-d6ec-4800-b76e-0131e7edffd4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:57.650903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1355/2671] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 18:59:57 2022] PUT /identity/v3/projects/1235f57770f34c87aeb3369fa8e04b54/users/1a5c2d856c8f4bfc9d59b976ca4371ee/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 18:59:57.659004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162ba4e-49c5-4f05-9595-a95e596bc107 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:57.659294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162ba4e-49c5-4f05-9595-a95e596bc107 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:57.659570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162ba4e-49c5-4f05-9595-a95e596bc107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:57.683829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-7162ba4e-49c5-4f05-9595-a95e596bc107 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:57.683829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7162ba4e-49c5-4f05-9595-a95e596bc107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:57.747206 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1317/2672] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:57.758127 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55effc-d6ec-4800-b76e-0131e7edffd4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:57.759918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-cf55effc-d6ec-4800-b76e-0131e7edffd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:57.821015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55effc-d6ec-4800-b76e-0131e7edffd4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:57.824116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5328a66-f29a-45fb-bc9c-add95ab9e857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:57.824758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5328a66-f29a-45fb-bc9c-add95ab9e857 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:57.825038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5328a66-f29a-45fb-bc9c-add95ab9e857 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:57.825337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5328a66-f29a-45fb-bc9c-add95ab9e857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:57.836986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5328a66-f29a-45fb-bc9c-add95ab9e857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:57.838451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5328a66-f29a-45fb-bc9c-add95ab9e857 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:57.848282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1356/2673] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 18:59:59.131085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dab4f-e96d-4119-a6ed-5a94015304c8 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:59.132168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dab4f-e96d-4119-a6ed-5a94015304c8 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:59.132337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dab4f-e96d-4119-a6ed-5a94015304c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:59.164517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-cf4dab4f-e96d-4119-a6ed-5a94015304c8 None None] MFA Rules not processed for user `1a5c2d856c8f4bfc9d59b976ca4371ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 18:59:59.165791 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cf4dab4f-e96d-4119-a6ed-5a94015304c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:59.234607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1318/2674] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 18:59:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 18:59:59.246104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5328a66-f29a-45fb-bc9c-add95ab9e857 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 18:59:59.247526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e5328a66-f29a-45fb-bc9c-add95ab9e857 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 18:59:59.306464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5328a66-f29a-45fb-bc9c-add95ab9e857 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 18:59:59.310093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9896a43-a774-49fd-a81c-00e94c281a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 18:59:59.310694 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9896a43-a774-49fd-a81c-00e94c281a4f service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 18:59:59.310977 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9896a43-a774-49fd-a81c-00e94c281a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 18:59:59.311250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9896a43-a774-49fd-a81c-00e94c281a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 18:59:59.320219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9896a43-a774-49fd-a81c-00e94c281a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 18:59:59.321683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9896a43-a774-49fd-a81c-00e94c281a4f service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 18:59:59.330544 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1357/2675] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 18:59:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:00:00.773994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4dab4f-e96d-4119-a6ed-5a94015304c8 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:00.775456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-cf4dab4f-e96d-4119-a6ed-5a94015304c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:00.789894 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4dab4f-e96d-4119-a6ed-5a94015304c8 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:00.794678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:00.795309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:00.795591 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:00.795900 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] PATH_INFO: `/v3/users/82dbaa4c11e348a3bc029693410adacb` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:00.797065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82dbaa4c11e348a3bc029693410adacb)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:00.799040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:00.966451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:00.971400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82dbaa4c11e348a3bc029693410adacb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:00.972040 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82dbaa4c11e348a3bc029693410adacb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:00.972642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82dbaa4c11e348a3bc029693410adacb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:00.985275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82dbaa4c11e348a3bc029693410adacb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:00.992775 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b90e9077-9ff8-4f85-baf2-8a818007ccb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82dbaa4c11e348a3bc029693410adacb'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:01.066821 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1319/2676] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:00:00 2022] DELETE /identity/v3/users/82dbaa4c11e348a3bc029693410adacb => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:01.072183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7656d77-aece-4819-9c55-63683e7b6c7f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:01.072658 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7656d77-aece-4819-9c55-63683e7b6c7f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:01.072777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7656d77-aece-4819-9c55-63683e7b6c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:01.083377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a62ad8-1781-47d2-b616-7757f05a9e42 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:01.083669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a62ad8-1781-47d2-b616-7757f05a9e42 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:01.083940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a62ad8-1781-47d2-b616-7757f05a9e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:01.092758 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c7656d77-aece-4819-9c55-63683e7b6c7f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:01.094014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c7656d77-aece-4819-9c55-63683e7b6c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:01.104545 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-35a62ad8-1781-47d2-b616-7757f05a9e42 None None] MFA Rules not processed for user `1a5c2d856c8f4bfc9d59b976ca4371ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:01.105943 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-35a62ad8-1781-47d2-b616-7757f05a9e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:01.155692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1358/2677] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:01.163264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1320/2678] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 19:00:01.166366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7656d77-aece-4819-9c55-63683e7b6c7f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:01.168130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c7656d77-aece-4819-9c55-63683e7b6c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:01.177962 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:01.230990 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7656d77-aece-4819-9c55-63683e7b6c7f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:01.234583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd13b90-6606-42d5-ab2d-b91c6c31628b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:01.235193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13b90-6606-42d5-ab2d-b91c6c31628b service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:01.235476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13b90-6606-42d5-ab2d-b91c6c31628b service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:01.235756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13b90-6606-42d5-ab2d-b91c6c31628b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:01.246651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd13b90-6606-42d5-ab2d-b91c6c31628b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:01.249874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd13b90-6606-42d5-ab2d-b91c6c31628b service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:01.258568 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1359/2679] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 19:00:01.469220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a62ad8-1781-47d2-b616-7757f05a9e42 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:01.479160 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a62ad8-1781-47d2-b616-7757f05a9e42 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:01.482655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e616729-0a72-4247-a080-97f6158e3ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:01.483254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e616729-0a72-4247-a080-97f6158e3ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:01.483551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e616729-0a72-4247-a080-97f6158e3ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:01.483812 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e616729-0a72-4247-a080-97f6158e3ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:01.485558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3e616729-0a72-4247-a080-97f6158e3ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:01.495584 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e616729-0a72-4247-a080-97f6158e3ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:01.496995 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e616729-0a72-4247-a080-97f6158e3ee1 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:01.504918 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1321/2680] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:00:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 19:00:01.911220 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd13b90-6606-42d5-ab2d-b91c6c31628b service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:01.912631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1fd13b90-6606-42d5-ab2d-b91c6c31628b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:01.928314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd13b90-6606-42d5-ab2d-b91c6c31628b service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:01.934015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:01.936513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:01.936513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:01.936513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] PATH_INFO: `/v3/projects/ac942a71f774453ab592e59618bf20a1` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:01.936513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac942a71f774453ab592e59618bf20a1)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:01.938497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:02.048349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac942a71f774453ab592e59618bf20a1', 'request_id': 'req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:02.060614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac942a71f774453ab592e59618bf20a1', 'request_id': 'req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:02.126692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac942a71f774453ab592e59618bf20a1', 'request_id': 'req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:02.138993 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] The token cache is being invalidate because project ac942a71f774453ab592e59618bf20a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:00:02.139291 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:02.140089 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:02.142498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1360/2681] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:00:01 2022] DELETE /identity/v3/projects/ac942a71f774453ab592e59618bf20a1 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:02.146003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e616729-0a72-4247-a080-97f6158e3ee1 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:02.158036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e616729-0a72-4247-a080-97f6158e3ee1 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:02.217563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:02.218857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:02.218857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:02.218857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] PATH_INFO: `/v3/users/a359ac4d7a94463dbf09c82f92c1f164` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:02.219796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a359ac4d7a94463dbf09c82f92c1f164)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:02.221758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:02.453137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a359ac4d7a94463dbf09c82f92c1f164'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:02.453962 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a359ac4d7a94463dbf09c82f92c1f164'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:02.454601 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a359ac4d7a94463dbf09c82f92c1f164'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:02.460390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:02.472524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a359ac4d7a94463dbf09c82f92c1f164'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:02.483541 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0e45e47a-1052-4749-a79c-10c0f324a9f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a359ac4d7a94463dbf09c82f92c1f164'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:02.567017 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1322/2682] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:00:02 2022] DELETE /identity/v3/users/a359ac4d7a94463dbf09c82f92c1f164 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:02.999464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:03.000905 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:03.016926 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ee9e3-0e8b-42ff-809e-28e132d13a81 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:03.078393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:03.079031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.079307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.079581 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] PATH_INFO: `/v3/projects/08c36b568df647cfb667db158b356c3c` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.080696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c36b568df647cfb667db158b356c3c)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:03.082902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:03.209035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:03.219102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c36b568df647cfb667db158b356c3c', 'request_id': 'req-4da5272f-887f-4169-82e7-e1a3eb62d408'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:03.225410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c36b568df647cfb667db158b356c3c', 'request_id': 'req-4da5272f-887f-4169-82e7-e1a3eb62d408'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:03.286817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c36b568df647cfb667db158b356c3c', 'request_id': 'req-4da5272f-887f-4169-82e7-e1a3eb62d408'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:03.298248 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] The token cache is being invalidate because project 08c36b568df647cfb667db158b356c3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:00:03.298541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:03.299247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-4da5272f-887f-4169-82e7-e1a3eb62d408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:03.300743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1361/2683] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:00:02 2022] DELETE /identity/v3/projects/08c36b568df647cfb667db158b356c3c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:03.306348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d3439-54de-4de9-9b53-c80442d9e133 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.306835 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d3439-54de-4de9-9b53-c80442d9e133 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.306955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d3439-54de-4de9-9b53-c80442d9e133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.327290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-7e0d3439-54de-4de9-9b53-c80442d9e133 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:03.328482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7e0d3439-54de-4de9-9b53-c80442d9e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:03.389153 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1323/2684] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:03.397480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43dfb2f-2aeb-4ced-8c5b-22700d5c26f5 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.397925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43dfb2f-2aeb-4ced-8c5b-22700d5c26f5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.405149 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43dfb2f-2aeb-4ced-8c5b-22700d5c26f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.419264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e43dfb2f-2aeb-4ced-8c5b-22700d5c26f5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:03.420487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e43dfb2f-2aeb-4ced-8c5b-22700d5c26f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:03.429806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1362/2685] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:03.435226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d3439-54de-4de9-9b53-c80442d9e133 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:03.448580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d3439-54de-4de9-9b53-c80442d9e133 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:03.454131 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0abd46-055f-422b-9236-639ac95eb6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:03.454719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0abd46-055f-422b-9236-639ac95eb6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.456709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0abd46-055f-422b-9236-639ac95eb6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.456709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0abd46-055f-422b-9236-639ac95eb6b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.456709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0abd46-055f-422b-9236-639ac95eb6b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 19:00:03.456997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0abd46-055f-422b-9236-639ac95eb6b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:03.459450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0abd46-055f-422b-9236-639ac95eb6b8 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:03.473673 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1324/2686] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 19:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 19:00:03.478549 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43dfb2f-2aeb-4ced-8c5b-22700d5c26f5 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:03.490005 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43dfb2f-2aeb-4ced-8c5b-22700d5c26f5 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:03.496433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd259f6-4877-4e28-9b90-97187809efc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:03.497035 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd259f6-4877-4e28-9b90-97187809efc0 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.497362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd259f6-4877-4e28-9b90-97187809efc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.497632 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd259f6-4877-4e28-9b90-97187809efc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.498870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd259f6-4877-4e28-9b90-97187809efc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:03.499347 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0abd46-055f-422b-9236-639ac95eb6b8 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:03.500752 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd259f6-4877-4e28-9b90-97187809efc0 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:03.501027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-7e0abd46-055f-422b-9236-639ac95eb6b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:03.589172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1363/2687] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:00:03 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:00:03.592364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd259f6-4877-4e28-9b90-97187809efc0 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:03.598499 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0abd46-055f-422b-9236-639ac95eb6b8 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:03.641083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:03.662244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4ee6f-d8e0-4e95-af89-e4588b6fb128 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:03.663372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4ee6f-d8e0-4e95-af89-e4588b6fb128 service placement] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.663917 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4ee6f-d8e0-4e95-af89-e4588b6fb128 service placement] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.664232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4ee6f-d8e0-4e95-af89-e4588b6fb128 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.666592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-64a4ee6f-d8e0-4e95-af89-e4588b6fb128 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:03.667201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd259f6-4877-4e28-9b90-97187809efc0 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:03.671902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719efba-4fa9-4e45-b3c1-336d5428bb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:03.672512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719efba-4fa9-4e45-b3c1-336d5428bb60 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.672799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719efba-4fa9-4e45-b3c1-336d5428bb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.673097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719efba-4fa9-4e45-b3c1-336d5428bb60 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.673733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719efba-4fa9-4e45-b3c1-336d5428bb60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:03.675634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719efba-4fa9-4e45-b3c1-336d5428bb60 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:03.678455 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4ee6f-d8e0-4e95-af89-e4588b6fb128 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:03.680128 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4ee6f-d8e0-4e95-af89-e4588b6fb128 service placement] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:03.691275 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1325/2688] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:00:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 19:00:03.755479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1364/2689] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:00:03 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:00:03.760700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4ee6f-d8e0-4e95-af89-e4588b6fb128 service placement] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:03.773636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4ee6f-d8e0-4e95-af89-e4588b6fb128 service placement] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:03.779914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e4a25-ac4b-4962-be8d-286e77e6e421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:03.780519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e4a25-ac4b-4962-be8d-286e77e6e421 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.780796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e4a25-ac4b-4962-be8d-286e77e6e421 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.781207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e4a25-ac4b-4962-be8d-286e77e6e421 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.781749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e4a25-ac4b-4962-be8d-286e77e6e421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:03.783333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e4a25-ac4b-4962-be8d-286e77e6e421 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:03.798456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1326/2690] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 19:00:03.802681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719efba-4fa9-4e45-b3c1-336d5428bb60 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:03.813374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719efba-4fa9-4e45-b3c1-336d5428bb60 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:03.818102 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a02c6-a156-47ab-a116-2f2eb5717a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:03.818701 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a02c6-a156-47ab-a116-2f2eb5717a67 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.819666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a02c6-a156-47ab-a116-2f2eb5717a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.819819 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a02c6-a156-47ab-a116-2f2eb5717a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.820413 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a02c6-a156-47ab-a116-2f2eb5717a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:03.822040 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a02c6-a156-47ab-a116-2f2eb5717a67 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:03.833036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1365/2691] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 19:00:03.838592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e4a25-ac4b-4962-be8d-286e77e6e421 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:03.848988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e4a25-ac4b-4962-be8d-286e77e6e421 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:03.854366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea214a79-24f8-439a-bf85-6cf4ee25a02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:03.855024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea214a79-24f8-439a-bf85-6cf4ee25a02a admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.855304 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea214a79-24f8-439a-bf85-6cf4ee25a02a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.855579 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea214a79-24f8-439a-bf85-6cf4ee25a02a admin admin] PATH_INFO: `/v3/projects/443e0b7cf8514b8d8dbb55a0ad718cd5/users/37885e786c6f49fab388dbae48484e0e/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.871184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea214a79-24f8-439a-bf85-6cf4ee25a02a admin admin] RBAC: Authorizing `identity:create_grant(project_id=443e0b7cf8514b8d8dbb55a0ad718cd5, user_id=37885e786c6f49fab388dbae48484e0e, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:03.873978 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea214a79-24f8-439a-bf85-6cf4ee25a02a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:03.947296 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1327/2692] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:00:03 2022] PUT /identity/v3/projects/443e0b7cf8514b8d8dbb55a0ad718cd5/users/37885e786c6f49fab388dbae48484e0e/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:03.959074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d37a4-3ae7-49b3-b983-d64ebec21919 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:03.959367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d37a4-3ae7-49b3-b983-d64ebec21919 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:03.959715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d37a4-3ae7-49b3-b983-d64ebec21919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:03.988092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4d5d37a4-3ae7-49b3-b983-d64ebec21919 None None] MFA Rules not processed for user `37885e786c6f49fab388dbae48484e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:03.989341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4d5d37a4-3ae7-49b3-b983-d64ebec21919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:04.061879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1366/2693] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 19:00:04.066560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea214a79-24f8-439a-bf85-6cf4ee25a02a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:04.135871 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea214a79-24f8-439a-bf85-6cf4ee25a02a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:04.140577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce21bb-ce73-4714-9b88-c37a6650c3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:04.141221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce21bb-ce73-4714-9b88-c37a6650c3a2 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:04.141504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce21bb-ce73-4714-9b88-c37a6650c3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:04.141779 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce21bb-ce73-4714-9b88-c37a6650c3a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:04.142448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce21bb-ce73-4714-9b88-c37a6650c3a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:04.144348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce21bb-ce73-4714-9b88-c37a6650c3a2 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:04.225446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1328/2694] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:00:04 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:00:04.229709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d37a4-3ae7-49b3-b983-d64ebec21919 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:04.235964 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:04.243026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d37a4-3ae7-49b3-b983-d64ebec21919 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:04.309524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f39bec-2940-4bcc-885a-a46afda0cfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:04.310133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f39bec-2940-4bcc-885a-a46afda0cfee admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:04.310411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f39bec-2940-4bcc-885a-a46afda0cfee admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:04.310684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f39bec-2940-4bcc-885a-a46afda0cfee admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:04.311308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f39bec-2940-4bcc-885a-a46afda0cfee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:04.313228 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f39bec-2940-4bcc-885a-a46afda0cfee admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:04.427640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1367/2695] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:00:04 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:00:04.432818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce21bb-ce73-4714-9b88-c37a6650c3a2 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:04.443827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce21bb-ce73-4714-9b88-c37a6650c3a2 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:04.448435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be6df3-167b-441c-913e-1690f1d0df20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:04.449364 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be6df3-167b-441c-913e-1690f1d0df20 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:04.449657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be6df3-167b-441c-913e-1690f1d0df20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:04.449933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be6df3-167b-441c-913e-1690f1d0df20 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:04.450724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be6df3-167b-441c-913e-1690f1d0df20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:04.452234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be6df3-167b-441c-913e-1690f1d0df20 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:04.463382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1329/2696] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:00:04.468101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f39bec-2940-4bcc-885a-a46afda0cfee admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:04.479172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f39bec-2940-4bcc-885a-a46afda0cfee admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:04.483892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593818e0-2e64-468a-bd04-452449984c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:04.484499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593818e0-2e64-468a-bd04-452449984c75 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:04.484782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593818e0-2e64-468a-bd04-452449984c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:04.485088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593818e0-2e64-468a-bd04-452449984c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:04.485744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593818e0-2e64-468a-bd04-452449984c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:04.487328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593818e0-2e64-468a-bd04-452449984c75 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:04.501092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1368/2697] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:00:04.502824 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be6df3-167b-441c-913e-1690f1d0df20 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:04.513075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be6df3-167b-441c-913e-1690f1d0df20 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:04.517692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e1339-9f76-40cf-ba8e-6258cd87ef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:04.518348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e1339-9f76-40cf-ba8e-6258cd87ef67 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:04.518632 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e1339-9f76-40cf-ba8e-6258cd87ef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:04.518913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e1339-9f76-40cf-ba8e-6258cd87ef67 admin admin] PATH_INFO: `/v3/projects/37497564e12e41eb9e6574c7cef9cf85/users/2df7667caa1945648a6638dc270e1a11/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:04.534381 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e1339-9f76-40cf-ba8e-6258cd87ef67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37497564e12e41eb9e6574c7cef9cf85, user_id=2df7667caa1945648a6638dc270e1a11, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:04.537031 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e1339-9f76-40cf-ba8e-6258cd87ef67 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:04.624442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1330/2698] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:00:04 2022] PUT /identity/v3/projects/37497564e12e41eb9e6574c7cef9cf85/users/2df7667caa1945648a6638dc270e1a11/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:04.691767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc69375-7137-4188-88d8-e1d636f08f39 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:04.691965 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc69375-7137-4188-88d8-e1d636f08f39 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:04.692236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc69375-7137-4188-88d8-e1d636f08f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:04.723281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-3fc69375-7137-4188-88d8-e1d636f08f39 None None] MFA Rules not processed for user `2df7667caa1945648a6638dc270e1a11`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:04.724936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-3fc69375-7137-4188-88d8-e1d636f08f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:04.794901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1369/2699] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:04.799933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e1339-9f76-40cf-ba8e-6258cd87ef67 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:04.854738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:04.871386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e1339-9f76-40cf-ba8e-6258cd87ef67 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:04.876102 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830899d-81d5-4f88-8d17-fb72cd84ec8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:04.876712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830899d-81d5-4f88-8d17-fb72cd84ec8d admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:04.876997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830899d-81d5-4f88-8d17-fb72cd84ec8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:04.877312 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830899d-81d5-4f88-8d17-fb72cd84ec8d admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:04.877949 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830899d-81d5-4f88-8d17-fb72cd84ec8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:04.879860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830899d-81d5-4f88-8d17-fb72cd84ec8d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:05.000284 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1331/2700] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:00:04 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:00:05.004162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc69375-7137-4188-88d8-e1d636f08f39 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:05.015135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc69375-7137-4188-88d8-e1d636f08f39 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:05.078657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab71f35-36de-4ea1-845c-6ceab40d5260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:05.079285 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab71f35-36de-4ea1-845c-6ceab40d5260 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:05.079571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab71f35-36de-4ea1-845c-6ceab40d5260 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:05.079978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab71f35-36de-4ea1-845c-6ceab40d5260 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:05.080641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab71f35-36de-4ea1-845c-6ceab40d5260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:05.082640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab71f35-36de-4ea1-845c-6ceab40d5260 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:05.176301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:05.182111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1370/2701] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:00:05 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:00:05.185975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830899d-81d5-4f88-8d17-fb72cd84ec8d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:05.200194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830899d-81d5-4f88-8d17-fb72cd84ec8d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:05.205105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55280665-0e2f-44af-82b8-abcb5402a363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:05.212712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55280665-0e2f-44af-82b8-abcb5402a363 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:05.212712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55280665-0e2f-44af-82b8-abcb5402a363 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:05.212712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55280665-0e2f-44af-82b8-abcb5402a363 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:05.212712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55280665-0e2f-44af-82b8-abcb5402a363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:05.212712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55280665-0e2f-44af-82b8-abcb5402a363 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:05.219413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1332/2702] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:00:05.223741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab71f35-36de-4ea1-845c-6ceab40d5260 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:05.234298 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab71f35-36de-4ea1-845c-6ceab40d5260 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:05.238909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c53e6-fbb3-429e-8472-c32238092340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:05.241411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c53e6-fbb3-429e-8472-c32238092340 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:05.241411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c53e6-fbb3-429e-8472-c32238092340 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:05.241411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c53e6-fbb3-429e-8472-c32238092340 admin admin] PATH_INFO: `/v3/projects/fe48367a79cd4d05bbb497e2dbcb79de/users/1eb19c7cc07a47cc8ea2ab28e49ae3bf/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:05.255738 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c53e6-fbb3-429e-8472-c32238092340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe48367a79cd4d05bbb497e2dbcb79de, user_id=1eb19c7cc07a47cc8ea2ab28e49ae3bf, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:05.259181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c53e6-fbb3-429e-8472-c32238092340 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:05.329540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1371/2703] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:00:05 2022] PUT /identity/v3/projects/fe48367a79cd4d05bbb497e2dbcb79de/users/1eb19c7cc07a47cc8ea2ab28e49ae3bf/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:05.333665 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55280665-0e2f-44af-82b8-abcb5402a363 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:05.344531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55280665-0e2f-44af-82b8-abcb5402a363 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:05.404688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f513385-7074-4e1f-8489-9ff5995a9252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:05.405332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513385-7074-4e1f-8489-9ff5995a9252 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:05.405636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513385-7074-4e1f-8489-9ff5995a9252 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:05.405907 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513385-7074-4e1f-8489-9ff5995a9252 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:05.406549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f513385-7074-4e1f-8489-9ff5995a9252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:05.408189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f513385-7074-4e1f-8489-9ff5995a9252 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:05.419852 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1333/2704] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:00:05.424366 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c53e6-fbb3-429e-8472-c32238092340 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:05.436302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c53e6-fbb3-429e-8472-c32238092340 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:05.441473 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80f49e-4123-4afb-b0d3-ebc86d19d5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:05.446947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80f49e-4123-4afb-b0d3-ebc86d19d5e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:05.446947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80f49e-4123-4afb-b0d3-ebc86d19d5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:05.446947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80f49e-4123-4afb-b0d3-ebc86d19d5e1 admin admin] PATH_INFO: `/v3/projects/fe48367a79cd4d05bbb497e2dbcb79de/users/1eb19c7cc07a47cc8ea2ab28e49ae3bf/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:05.446947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80f49e-4123-4afb-b0d3-ebc86d19d5e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe48367a79cd4d05bbb497e2dbcb79de, user_id=1eb19c7cc07a47cc8ea2ab28e49ae3bf, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:05.448288 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80f49e-4123-4afb-b0d3-ebc86d19d5e1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:05.460279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1372/2705] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:00:05 2022] PUT /identity/v3/projects/fe48367a79cd4d05bbb497e2dbcb79de/users/1eb19c7cc07a47cc8ea2ab28e49ae3bf/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:05.471194 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2c15d-1ca8-4295-bb02-dff5606aa4cd None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:05.479122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2c15d-1ca8-4295-bb02-dff5606aa4cd None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:05.479122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2c15d-1ca8-4295-bb02-dff5606aa4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:05.503906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f1e2c15d-1ca8-4295-bb02-dff5606aa4cd None None] MFA Rules not processed for user `1eb19c7cc07a47cc8ea2ab28e49ae3bf`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:05.505198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f1e2c15d-1ca8-4295-bb02-dff5606aa4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:05.577859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1334/2706] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:05.590031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80f49e-4123-4afb-b0d3-ebc86d19d5e1 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:05.591786 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7f80f49e-4123-4afb-b0d3-ebc86d19d5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:05.655181 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80f49e-4123-4afb-b0d3-ebc86d19d5e1 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:05.658897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b99591-e338-4ec1-9073-d8c39f138bfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:05.659519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b99591-e338-4ec1-9073-d8c39f138bfa service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:05.659799 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b99591-e338-4ec1-9073-d8c39f138bfa service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:05.660075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b99591-e338-4ec1-9073-d8c39f138bfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:05.669655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b99591-e338-4ec1-9073-d8c39f138bfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:05.671078 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b99591-e338-4ec1-9073-d8c39f138bfa service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:05.681863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1373/2707] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:00:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:00:05.784437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2c15d-1ca8-4295-bb02-dff5606aa4cd None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:05.785861 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f1e2c15d-1ca8-4295-bb02-dff5606aa4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:05.798637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2c15d-1ca8-4295-bb02-dff5606aa4cd None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:05.802328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfd1e6-19c9-47a6-b9ea-e1e3e3466f22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:05.802937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd1e6-19c9-47a6-b9ea-e1e3e3466f22 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:05.803217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd1e6-19c9-47a6-b9ea-e1e3e3466f22 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:05.803549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfd1e6-19c9-47a6-b9ea-e1e3e3466f22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:05.851299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:05.872617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfd1e6-19c9-47a6-b9ea-e1e3e3466f22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:05.874224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfd1e6-19c9-47a6-b9ea-e1e3e3466f22 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:05.884830 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1335/2708] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:00:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:00:09.858050 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b99591-e338-4ec1-9073-d8c39f138bfa service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:09.924891 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b99591-e338-4ec1-9073-d8c39f138bfa service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:09.929813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:09.930471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:09.930997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:09.931557 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] PATH_INFO: `/v3/users/37885e786c6f49fab388dbae48484e0e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:09.932485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=37885e786c6f49fab388dbae48484e0e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:09.934384 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:10.156675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37885e786c6f49fab388dbae48484e0e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:10.157379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37885e786c6f49fab388dbae48484e0e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:10.158026 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37885e786c6f49fab388dbae48484e0e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:10.166033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37885e786c6f49fab388dbae48484e0e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:10.174893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:10.178911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37885e786c6f49fab388dbae48484e0e'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:10.254500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1374/2709] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:00:09 2022] DELETE /identity/v3/users/37885e786c6f49fab388dbae48484e0e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:10.259580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061a983-a11f-4aed-85a3-1b41d96715db None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:10.260034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061a983-a11f-4aed-85a3-1b41d96715db None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:10.260720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061a983-a11f-4aed-85a3-1b41d96715db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:10.280223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-5061a983-a11f-4aed-85a3-1b41d96715db None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:10.289571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5061a983-a11f-4aed-85a3-1b41d96715db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:10.343292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1336/2710] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:10.352923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:10.354389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:10.414661 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbe136-ac95-4190-9edb-4b74a4ec4dfd admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:10.418588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b995-6b0f-4f4d-910f-a67d576ed668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:10.419221 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b995-6b0f-4f4d-910f-a67d576ed668 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:10.419501 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b995-6b0f-4f4d-910f-a67d576ed668 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:10.419775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b995-6b0f-4f4d-910f-a67d576ed668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:10.429315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16b995-6b0f-4f4d-910f-a67d576ed668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:10.430800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16b995-6b0f-4f4d-910f-a67d576ed668 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:10.441227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1375/2711] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:00:10.862038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061a983-a11f-4aed-85a3-1b41d96715db None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:10.868692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5061a983-a11f-4aed-85a3-1b41d96715db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:10.877863 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061a983-a11f-4aed-85a3-1b41d96715db None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:10.882641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:10.883276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:10.883555 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:10.884445 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] PATH_INFO: `/v3/projects/443e0b7cf8514b8d8dbb55a0ad718cd5` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:10.885827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=443e0b7cf8514b8d8dbb55a0ad718cd5)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:10.887738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:10.990709 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '443e0b7cf8514b8d8dbb55a0ad718cd5', 'request_id': 'req-8d57858c-0fcf-4ac3-8336-930e4414c839'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:10.997313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '443e0b7cf8514b8d8dbb55a0ad718cd5', 'request_id': 'req-8d57858c-0fcf-4ac3-8336-930e4414c839'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:11.002926 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '443e0b7cf8514b8d8dbb55a0ad718cd5', 'request_id': 'req-8d57858c-0fcf-4ac3-8336-930e4414c839'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:11.065313 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] The token cache is being invalidate because project 443e0b7cf8514b8d8dbb55a0ad718cd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:00:11.065603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:11.066224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:11.067700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1337/2712] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:00:10 2022] DELETE /identity/v3/projects/443e0b7cf8514b8d8dbb55a0ad718cd5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:11.072409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b995-6b0f-4f4d-910f-a67d576ed668 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:11.082197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b995-6b0f-4f4d-910f-a67d576ed668 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:11.144543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:11.145219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:11.145502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:11.145777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] PATH_INFO: `/v3/users/2df7667caa1945648a6638dc270e1a11` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:11.146922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2df7667caa1945648a6638dc270e1a11)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:11.148966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:11.346512 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2df7667caa1945648a6638dc270e1a11'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:11.347170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2df7667caa1945648a6638dc270e1a11'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:11.347777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2df7667caa1945648a6638dc270e1a11'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:11.355640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2df7667caa1945648a6638dc270e1a11'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:11.367781 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2df7667caa1945648a6638dc270e1a11'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:11.454810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1376/2713] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:00:11 2022] DELETE /identity/v3/users/2df7667caa1945648a6638dc270e1a11 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:11.918891 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:11.920363 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:11.934549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57858c-0fcf-4ac3-8336-930e4414c839 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:11.996685 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:11.997514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:11.997931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:11.998311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] PATH_INFO: `/v3/projects/37497564e12e41eb9e6574c7cef9cf85` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:11.999606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37497564e12e41eb9e6574c7cef9cf85)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:12.001670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:12.004130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:12.121670 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37497564e12e41eb9e6574c7cef9cf85', 'request_id': 'req-101c8d80-d502-4730-b79a-77687ff080e6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:12.128056 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37497564e12e41eb9e6574c7cef9cf85', 'request_id': 'req-101c8d80-d502-4730-b79a-77687ff080e6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:12.134028 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37497564e12e41eb9e6574c7cef9cf85', 'request_id': 'req-101c8d80-d502-4730-b79a-77687ff080e6'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:12.183984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] The token cache is being invalidate because project 37497564e12e41eb9e6574c7cef9cf85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:00:12.184236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:12.184933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:12.186680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1338/2714] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:00:11 2022] DELETE /identity/v3/projects/37497564e12e41eb9e6574c7cef9cf85 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:12.190515 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:12.192016 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:12.207823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1850c-1f2c-4139-b688-4ef0e77bd93c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:12.212196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:12.267452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:12.268081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:12.268361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:12.268631 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] PATH_INFO: `/v3/users/1eb19c7cc07a47cc8ea2ab28e49ae3bf` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:12.269767 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb19c7cc07a47cc8ea2ab28e49ae3bf)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:12.271759 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:12.518059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb19c7cc07a47cc8ea2ab28e49ae3bf'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:12.518723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb19c7cc07a47cc8ea2ab28e49ae3bf'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:12.519415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb19c7cc07a47cc8ea2ab28e49ae3bf'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:12.526456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb19c7cc07a47cc8ea2ab28e49ae3bf'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:12.538921 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-cbdfa2b0-f842-418b-a380-ad3ccb490611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb19c7cc07a47cc8ea2ab28e49ae3bf'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:12.613104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1377/2715] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:00:12 2022] DELETE /identity/v3/users/1eb19c7cc07a47cc8ea2ab28e49ae3bf => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 19:00:12.999527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:13.000983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:13.014183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c8d80-d502-4730-b79a-77687ff080e6 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:13.072899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:13.073567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:13.073871 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:13.074230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] PATH_INFO: `/v3/projects/fe48367a79cd4d05bbb497e2dbcb79de` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:13.075389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe48367a79cd4d05bbb497e2dbcb79de)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:13.077338 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:13.210043 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe48367a79cd4d05bbb497e2dbcb79de', 'request_id': 'req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:13.216928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe48367a79cd4d05bbb497e2dbcb79de', 'request_id': 'req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:13.222513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe48367a79cd4d05bbb497e2dbcb79de', 'request_id': 'req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:13.305440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] The token cache is being invalidate because project fe48367a79cd4d05bbb497e2dbcb79de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:00:13.305729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:13.306369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6610f48a-cfa8-4ab1-b4d8-4c73d91172d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:13.307840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1339/2716] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:00:12 2022] DELETE /identity/v3/projects/fe48367a79cd4d05bbb497e2dbcb79de => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:13.314476 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81227422-cadc-4dd5-87ac-9d92b8d373c6 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:13.314978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81227422-cadc-4dd5-87ac-9d92b8d373c6 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:13.315097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81227422-cadc-4dd5-87ac-9d92b8d373c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:13.335334 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-81227422-cadc-4dd5-87ac-9d92b8d373c6 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:13.336551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-81227422-cadc-4dd5-87ac-9d92b8d373c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:13.398328 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1378/2717] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:13.408503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ee63d-d47c-4ea5-899a-1e26a9836248 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:13.408792 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ee63d-d47c-4ea5-899a-1e26a9836248 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:13.409140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ee63d-d47c-4ea5-899a-1e26a9836248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:13.430121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-8a6ee63d-d47c-4ea5-899a-1e26a9836248 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:13.431382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8a6ee63d-d47c-4ea5-899a-1e26a9836248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:13.439822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1340/2718] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:13.444183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81227422-cadc-4dd5-87ac-9d92b8d373c6 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:13.454432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81227422-cadc-4dd5-87ac-9d92b8d373c6 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:13.458971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ab94b-f262-4955-8d00-a291535b24a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:13.459603 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ab94b-f262-4955-8d00-a291535b24a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:13.459890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ab94b-f262-4955-8d00-a291535b24a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:13.460170 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ab94b-f262-4955-8d00-a291535b24a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:13.460897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ab94b-f262-4955-8d00-a291535b24a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 19:00:13.461241 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ab94b-f262-4955-8d00-a291535b24a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:13.462862 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ab94b-f262-4955-8d00-a291535b24a2 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:13.479591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1379/2719] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 19:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 19:00:13.486382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ee63d-d47c-4ea5-899a-1e26a9836248 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:13.491482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ee63d-d47c-4ea5-899a-1e26a9836248 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:13.497796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508a78-3aa0-4422-b295-1d9660698854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:13.498401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508a78-3aa0-4422-b295-1d9660698854 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:13.498686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508a78-3aa0-4422-b295-1d9660698854 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:13.498961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508a78-3aa0-4422-b295-1d9660698854 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:13.499590 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84508a78-3aa0-4422-b295-1d9660698854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:13.501530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84508a78-3aa0-4422-b295-1d9660698854 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:13.517495 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:13.612295 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1341/2720] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:00:13 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:00:13.616755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ab94b-f262-4955-8d00-a291535b24a2 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:13.628416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ab94b-f262-4955-8d00-a291535b24a2 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:13.637861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:13.685329 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8464e63-12e3-4723-83f5-e0ac85b5272c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:13.685952 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8464e63-12e3-4723-83f5-e0ac85b5272c admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:13.686239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8464e63-12e3-4723-83f5-e0ac85b5272c admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:13.686518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8464e63-12e3-4723-83f5-e0ac85b5272c admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:13.687193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8464e63-12e3-4723-83f5-e0ac85b5272c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:13.689137 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8464e63-12e3-4723-83f5-e0ac85b5272c admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:13.803897 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1380/2721] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:00:13 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:00:13.808871 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508a78-3aa0-4422-b295-1d9660698854 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:13.821170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508a78-3aa0-4422-b295-1d9660698854 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:13.827592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3816bb-6b7c-4d87-9391-c738677341dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:13.828193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3816bb-6b7c-4d87-9391-c738677341dd admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:13.828471 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3816bb-6b7c-4d87-9391-c738677341dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:13.828742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3816bb-6b7c-4d87-9391-c738677341dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:13.829418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3816bb-6b7c-4d87-9391-c738677341dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:13.831008 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3816bb-6b7c-4d87-9391-c738677341dd admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:13.841229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1342/2722] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:00:13.845666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8464e63-12e3-4723-83f5-e0ac85b5272c admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:13.856321 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8464e63-12e3-4723-83f5-e0ac85b5272c admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:13.861309 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029f9d8-3a08-4978-9ac6-58d4488df2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:13.861900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029f9d8-3a08-4978-9ac6-58d4488df2bc admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:13.862179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029f9d8-3a08-4978-9ac6-58d4488df2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:13.862452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029f9d8-3a08-4978-9ac6-58d4488df2bc admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:13.863086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029f9d8-3a08-4978-9ac6-58d4488df2bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:13.864647 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029f9d8-3a08-4978-9ac6-58d4488df2bc admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:13.876462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1381/2723] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:00:13.880563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3816bb-6b7c-4d87-9391-c738677341dd admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:13.891278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3816bb-6b7c-4d87-9391-c738677341dd admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:13.895914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d413d58-51b3-430c-8405-0e8ce9df3017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:13.896558 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d413d58-51b3-430c-8405-0e8ce9df3017 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:13.896834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d413d58-51b3-430c-8405-0e8ce9df3017 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:13.900834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d413d58-51b3-430c-8405-0e8ce9df3017 admin admin] PATH_INFO: `/v3/projects/b331aba645004da7b3f0a73127ffe482/users/64ced98298314052903edbca1e66cbca/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:13.911699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d413d58-51b3-430c-8405-0e8ce9df3017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b331aba645004da7b3f0a73127ffe482, user_id=64ced98298314052903edbca1e66cbca, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:13.914450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d413d58-51b3-430c-8405-0e8ce9df3017 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:14.001648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1343/2724] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:00:13 2022] PUT /identity/v3/projects/b331aba645004da7b3f0a73127ffe482/users/64ced98298314052903edbca1e66cbca/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 19:00:14.013282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf7bb-1cae-4400-9eed-7559cc62475b None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:14.013570 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf7bb-1cae-4400-9eed-7559cc62475b None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:14.013848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf7bb-1cae-4400-9eed-7559cc62475b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:14.045246 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c06cf7bb-1cae-4400-9eed-7559cc62475b None None] MFA Rules not processed for user `64ced98298314052903edbca1e66cbca`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:14.046475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c06cf7bb-1cae-4400-9eed-7559cc62475b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:14.110608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1382/2725] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:14.122229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d413d58-51b3-430c-8405-0e8ce9df3017 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:14.124154 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6d413d58-51b3-430c-8405-0e8ce9df3017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:14.180804 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d413d58-51b3-430c-8405-0e8ce9df3017 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:14.184748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d153f40-652b-446b-bc77-e470fb42199a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:14.185375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d153f40-652b-446b-bc77-e470fb42199a service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:14.185680 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d153f40-652b-446b-bc77-e470fb42199a service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:14.185953 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d153f40-652b-446b-bc77-e470fb42199a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:14.194106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d153f40-652b-446b-bc77-e470fb42199a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:14.195528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d153f40-652b-446b-bc77-e470fb42199a service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:14.205069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1344/2726] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:00:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:00:14.229394 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cf7bb-1cae-4400-9eed-7559cc62475b None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:14.291668 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cf7bb-1cae-4400-9eed-7559cc62475b None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:14.296947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:14.298109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:14.298109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:14.298109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] PATH_INFO: `/v3/users/64ced98298314052903edbca1e66cbca` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:14.299233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ced98298314052903edbca1e66cbca)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:14.301158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:14.324577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:14.491969 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ced98298314052903edbca1e66cbca'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:14.492558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ced98298314052903edbca1e66cbca'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:14.493262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ced98298314052903edbca1e66cbca'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:14.500666 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ced98298314052903edbca1e66cbca'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:14.513100 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ced98298314052903edbca1e66cbca'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:14.612929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1383/2727] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:00:14 2022] DELETE /identity/v3/users/64ced98298314052903edbca1e66cbca => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:14.617678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d969184-5cae-4c25-bec5-51520bcb6498 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:14.617961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d969184-5cae-4c25-bec5-51520bcb6498 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:14.618233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d969184-5cae-4c25-bec5-51520bcb6498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:14.637685 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-6d969184-5cae-4c25-bec5-51520bcb6498 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:14.638885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6d969184-5cae-4c25-bec5-51520bcb6498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:14.701980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1345/2728] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:14.711449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:14.712865 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:14.774260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3521d-1c21-40ee-ad13-699cfacf80e5 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:14.778480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22843e5c-a69c-4f5a-b24b-741a337b246b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:14.779094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22843e5c-a69c-4f5a-b24b-741a337b246b service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:14.779375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22843e5c-a69c-4f5a-b24b-741a337b246b service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:14.779652 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22843e5c-a69c-4f5a-b24b-741a337b246b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:14.789073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22843e5c-a69c-4f5a-b24b-741a337b246b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:14.790497 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22843e5c-a69c-4f5a-b24b-741a337b246b service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:14.801623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1384/2729] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:00:15.225770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d969184-5cae-4c25-bec5-51520bcb6498 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:15.227260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6d969184-5cae-4c25-bec5-51520bcb6498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:15.240389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d969184-5cae-4c25-bec5-51520bcb6498 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:15.245690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:15.247062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:15.247490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:15.247833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] PATH_INFO: `/v3/projects/b331aba645004da7b3f0a73127ffe482` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:15.249152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b331aba645004da7b3f0a73127ffe482)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:15.251124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:15.363929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b331aba645004da7b3f0a73127ffe482', 'request_id': 'req-5a316e10-c75b-4a5b-876a-1ce9097e924d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:15.372041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b331aba645004da7b3f0a73127ffe482', 'request_id': 'req-5a316e10-c75b-4a5b-876a-1ce9097e924d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:15.374306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:15.378276 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b331aba645004da7b3f0a73127ffe482', 'request_id': 'req-5a316e10-c75b-4a5b-876a-1ce9097e924d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:15.471783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] The token cache is being invalidate because project b331aba645004da7b3f0a73127ffe482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:00:15.472094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:15.478076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-5a316e10-c75b-4a5b-876a-1ce9097e924d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:15.478076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1346/2730] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:00:15 2022] DELETE /identity/v3/projects/b331aba645004da7b3f0a73127ffe482 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:15.482196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a7710-1096-4950-877e-0432a8e69ff5 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:15.482664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a7710-1096-4950-877e-0432a8e69ff5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:15.482913 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a7710-1096-4950-877e-0432a8e69ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:15.504409 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-290a7710-1096-4950-877e-0432a8e69ff5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:15.505696 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-290a7710-1096-4950-877e-0432a8e69ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:15.571315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1385/2731] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 19:00:15.579270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030a026-80f1-4c5e-a0da-23017e833869 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:15.579563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030a026-80f1-4c5e-a0da-23017e833869 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:15.579846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030a026-80f1-4c5e-a0da-23017e833869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:15.600404 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-f030a026-80f1-4c5e-a0da-23017e833869 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:15.601629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f030a026-80f1-4c5e-a0da-23017e833869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:15.610283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1347/2732] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:15.614542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a7710-1096-4950-877e-0432a8e69ff5 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:15.625196 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a7710-1096-4950-877e-0432a8e69ff5 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:15.630369 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b64dd1-ede0-4860-b73f-822d9c29fe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:15.630971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b64dd1-ede0-4860-b73f-822d9c29fe1b admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:15.631249 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b64dd1-ede0-4860-b73f-822d9c29fe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:15.631520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b64dd1-ede0-4860-b73f-822d9c29fe1b admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:15.632161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b64dd1-ede0-4860-b73f-822d9c29fe1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 19:00:15.632463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b64dd1-ede0-4860-b73f-822d9c29fe1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:15.634069 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b64dd1-ede0-4860-b73f-822d9c29fe1b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:15.645612 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1386/2733] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 19:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 19:00:15.650983 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030a026-80f1-4c5e-a0da-23017e833869 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:15.659660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030a026-80f1-4c5e-a0da-23017e833869 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:15.664212 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c5c75-9c6c-4cba-a20f-b0fa15b3b4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:15.664814 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c5c75-9c6c-4cba-a20f-b0fa15b3b4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:15.665121 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c5c75-9c6c-4cba-a20f-b0fa15b3b4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:15.665400 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c5c75-9c6c-4cba-a20f-b0fa15b3b4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:15.666039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c5c75-9c6c-4cba-a20f-b0fa15b3b4b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:15.668480 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c5c75-9c6c-4cba-a20f-b0fa15b3b4b4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:15.757117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1348/2734] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:00:15 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:00:15.762004 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b64dd1-ede0-4860-b73f-822d9c29fe1b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:15.772985 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b64dd1-ede0-4860-b73f-822d9c29fe1b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:15.803150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:15.831558 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cecd5-d93a-406e-9794-94af804e6264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:15.832171 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cecd5-d93a-406e-9794-94af804e6264 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:15.832453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cecd5-d93a-406e-9794-94af804e6264 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:15.832730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cecd5-d93a-406e-9794-94af804e6264 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:15.833405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cecd5-d93a-406e-9794-94af804e6264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:15.835308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cecd5-d93a-406e-9794-94af804e6264 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:15.910124 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1387/2735] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:00:15 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:00:15.914308 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c5c75-9c6c-4cba-a20f-b0fa15b3b4b4 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:15.924725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c5c75-9c6c-4cba-a20f-b0fa15b3b4b4 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:15.929536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a972768-463d-4dbf-b6ff-a465c380fa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:15.930137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a972768-463d-4dbf-b6ff-a465c380fa6c admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:15.930422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a972768-463d-4dbf-b6ff-a465c380fa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:15.930697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a972768-463d-4dbf-b6ff-a465c380fa6c admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:15.931340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a972768-463d-4dbf-b6ff-a465c380fa6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:15.932935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a972768-463d-4dbf-b6ff-a465c380fa6c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:15.946348 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1349/2736] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:00:15.947485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cecd5-d93a-406e-9794-94af804e6264 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:15.958048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cecd5-d93a-406e-9794-94af804e6264 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:15.962802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd0a0b-13e7-42a9-8126-1d2a55a9f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:15.963402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd0a0b-13e7-42a9-8126-1d2a55a9f95b admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:15.963685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd0a0b-13e7-42a9-8126-1d2a55a9f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:15.963961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd0a0b-13e7-42a9-8126-1d2a55a9f95b admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:15.964717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd0a0b-13e7-42a9-8126-1d2a55a9f95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:15.966352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd0a0b-13e7-42a9-8126-1d2a55a9f95b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:15.976208 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1388/2737] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 19:00:15.980431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a972768-463d-4dbf-b6ff-a465c380fa6c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:15.990645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a972768-463d-4dbf-b6ff-a465c380fa6c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:15.996509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485a046-4b6e-473f-b1bf-785309e51f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:15.997248 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485a046-4b6e-473f-b1bf-785309e51f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:15.997540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485a046-4b6e-473f-b1bf-785309e51f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:15.997815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485a046-4b6e-473f-b1bf-785309e51f2e admin admin] PATH_INFO: `/v3/projects/84398f99ba3e4244b06ee667c05c5d52/users/eb6bbff69c364c06aa057be9c816f094/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:16.014755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485a046-4b6e-473f-b1bf-785309e51f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=84398f99ba3e4244b06ee667c05c5d52, user_id=eb6bbff69c364c06aa057be9c816f094, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:16.017524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485a046-4b6e-473f-b1bf-785309e51f2e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:16.103270 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1350/2738] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:00:15 2022] PUT /identity/v3/projects/84398f99ba3e4244b06ee667c05c5d52/users/eb6bbff69c364c06aa057be9c816f094/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 19:00:16.114850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7da75-85e3-4603-9e0a-344a6c80670a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:16.115068 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7da75-85e3-4603-9e0a-344a6c80670a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:16.115341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7da75-85e3-4603-9e0a-344a6c80670a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:16.145522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-27f7da75-85e3-4603-9e0a-344a6c80670a None None] MFA Rules not processed for user `eb6bbff69c364c06aa057be9c816f094`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:16.146937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-27f7da75-85e3-4603-9e0a-344a6c80670a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:16.215108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1389/2739] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:16.226047 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485a046-4b6e-473f-b1bf-785309e51f2e admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:16.227885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f485a046-4b6e-473f-b1bf-785309e51f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:16.288976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485a046-4b6e-473f-b1bf-785309e51f2e admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:16.293145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98cebc-43fa-4a3c-8ca1-cd1c960160f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:16.299929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98cebc-43fa-4a3c-8ca1-cd1c960160f7 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:16.299929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98cebc-43fa-4a3c-8ca1-cd1c960160f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:16.299929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98cebc-43fa-4a3c-8ca1-cd1c960160f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:16.302707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98cebc-43fa-4a3c-8ca1-cd1c960160f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:16.304136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98cebc-43fa-4a3c-8ca1-cd1c960160f7 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:16.313766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1351/2740] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:00:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:00:18.042001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627140a-fd40-4d1d-9bed-f51db223dd7e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:18.042493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627140a-fd40-4d1d-9bed-f51db223dd7e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:18.042588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627140a-fd40-4d1d-9bed-f51db223dd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:18.062755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c627140a-fd40-4d1d-9bed-f51db223dd7e None None] MFA Rules not processed for user `eb6bbff69c364c06aa057be9c816f094`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:18.063975 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c627140a-fd40-4d1d-9bed-f51db223dd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:18.071227 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1390/2741] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:26.309951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=57,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:26.985061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab69ac-1024-4ada-9298-e9e76b79e0f5 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:26.986813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab69ac-1024-4ada-9298-e9e76b79e0f5 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:26.986813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab69ac-1024-4ada-9298-e9e76b79e0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:27.005269 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-a0ab69ac-1024-4ada-9298-e9e76b79e0f5 None None] MFA Rules not processed for user `1c04b545aa674beab518178a8936b788`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:27.006473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a0ab69ac-1024-4ada-9298-e9e76b79e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:27.060832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1352/2742] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:28.056488 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:37.054123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=25 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:38.173099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24819374-066f-4d5d-a8fe-e19dacda8951 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:38.173099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24819374-066f-4d5d-a8fe-e19dacda8951 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:38.173608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24819374-066f-4d5d-a8fe-e19dacda8951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:38.214394 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-24819374-066f-4d5d-a8fe-e19dacda8951 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:38.215079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-24819374-066f-4d5d-a8fe-e19dacda8951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:38.283434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1391/2743] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 19:00:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:38.423274 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab69ac-1024-4ada-9298-e9e76b79e0f5 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:38.426122 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a0ab69ac-1024-4ada-9298-e9e76b79e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:38.501920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab69ac-1024-4ada-9298-e9e76b79e0f5 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:38.507097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbf2dd-c94b-4b30-ba0a-de2f0f8be465 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:38.507704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf2dd-c94b-4b30-ba0a-de2f0f8be465 service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:38.507981 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf2dd-c94b-4b30-ba0a-de2f0f8be465 service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:38.508252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbf2dd-c94b-4b30-ba0a-de2f0f8be465 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:38.516897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cbf2dd-c94b-4b30-ba0a-de2f0f8be465 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:38.518551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cbf2dd-c94b-4b30-ba0a-de2f0f8be465 service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:38.527015 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1353/2744] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 19:00:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 19:00:39.316936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46709e46-f21a-4ab4-a0f2-7ce7c01c28ef None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:39.317443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46709e46-f21a-4ab4-a0f2-7ce7c01c28ef None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:39.317542 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46709e46-f21a-4ab4-a0f2-7ce7c01c28ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:39.337888 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-46709e46-f21a-4ab4-a0f2-7ce7c01c28ef None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:39.339108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-46709e46-f21a-4ab4-a0f2-7ce7c01c28ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:39.346665 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1392/2745] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 19:00:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:39.359324 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbf2dd-c94b-4b30-ba0a-de2f0f8be465 service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:39.370401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbf2dd-c94b-4b30-ba0a-de2f0f8be465 service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:39.376032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837a6d3-2a59-4229-820f-d13f5b0d9a82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:39.377307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837a6d3-2a59-4229-820f-d13f5b0d9a82 service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:39.377307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837a6d3-2a59-4229-820f-d13f5b0d9a82 service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:39.377307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837a6d3-2a59-4229-820f-d13f5b0d9a82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:39.385185 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837a6d3-2a59-4229-820f-d13f5b0d9a82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:39.386744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837a6d3-2a59-4229-820f-d13f5b0d9a82 service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:39.395443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1354/2746] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 19:00:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 19:00:40.026657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8842b-e3d4-43b3-911d-0cffd27df1a0 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:40.027020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8842b-e3d4-43b3-911d-0cffd27df1a0 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:40.027362 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8842b-e3d4-43b3-911d-0cffd27df1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:40.052503 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-8ae8842b-e3d4-43b3-911d-0cffd27df1a0 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:40.053779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8ae8842b-e3d4-43b3-911d-0cffd27df1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:40.061469 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1393/2747] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 19:00:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:40.076203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837a6d3-2a59-4229-820f-d13f5b0d9a82 service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:40.084882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837a6d3-2a59-4229-820f-d13f5b0d9a82 service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:40.090602 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58a738-c2c1-4574-8a07-cebafdd35009 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:40.091842 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58a738-c2c1-4574-8a07-cebafdd35009 service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:40.092133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58a738-c2c1-4574-8a07-cebafdd35009 service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:40.092414 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58a738-c2c1-4574-8a07-cebafdd35009 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:40.105956 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58a738-c2c1-4574-8a07-cebafdd35009 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:40.108388 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58a738-c2c1-4574-8a07-cebafdd35009 service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:40.116950 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1355/2748] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 19:00:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 19:00:40.666662 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2011106-a85d-4400-b165-cbc45bdc12e5 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:40.667129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2011106-a85d-4400-b165-cbc45bdc12e5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:40.667245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2011106-a85d-4400-b165-cbc45bdc12e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:40.687045 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-a2011106-a85d-4400-b165-cbc45bdc12e5 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:40.688171 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a2011106-a85d-4400-b165-cbc45bdc12e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:40.695418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1394/2749] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 19:00:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:40.708834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58a738-c2c1-4574-8a07-cebafdd35009 service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:40.719725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58a738-c2c1-4574-8a07-cebafdd35009 service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:40.724738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45e800-6edb-4078-bf71-c36916c69cb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:40.725609 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45e800-6edb-4078-bf71-c36916c69cb4 service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:40.725887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45e800-6edb-4078-bf71-c36916c69cb4 service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:40.726157 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45e800-6edb-4078-bf71-c36916c69cb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:40.734568 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45e800-6edb-4078-bf71-c36916c69cb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:40.736227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45e800-6edb-4078-bf71-c36916c69cb4 service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:40.744382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1356/2750] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 19:00:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 19:00:45.654992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2011106-a85d-4400-b165-cbc45bdc12e5 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:45.719175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2011106-a85d-4400-b165-cbc45bdc12e5 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:45.723667 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:45.724292 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:45.724569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:45.724839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] PATH_INFO: `/v3/users/eb6bbff69c364c06aa057be9c816f094` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:45.725998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb6bbff69c364c06aa057be9c816f094)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:45.727907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:45.901999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb6bbff69c364c06aa057be9c816f094'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:45.902671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb6bbff69c364c06aa057be9c816f094'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:45.903820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb6bbff69c364c06aa057be9c816f094'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:45.911842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb6bbff69c364c06aa057be9c816f094'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:45.923955 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:45.925381 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb6bbff69c364c06aa057be9c816f094'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:46.014571 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1395/2751] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:00:45 2022] DELETE /identity/v3/users/eb6bbff69c364c06aa057be9c816f094 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:00:46.019182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a1f07-5c35-4ff7-8393-a8865b6b8657 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:46.019662 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a1f07-5c35-4ff7-8393-a8865b6b8657 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:46.019761 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a1f07-5c35-4ff7-8393-a8865b6b8657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:46.039613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ea0a1f07-5c35-4ff7-8393-a8865b6b8657 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:00:46.040810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ea0a1f07-5c35-4ff7-8393-a8865b6b8657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:46.099576 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1357/2752] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:00:46.110000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:46.111502 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:46.169472 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189031a5-0e5b-4713-b1a1-b82293000b90 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:46.173221 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ea2f0-7b0b-466a-9aff-40ef0cab8e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:46.173829 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ea2f0-7b0b-466a-9aff-40ef0cab8e34 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:46.174104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ea2f0-7b0b-466a-9aff-40ef0cab8e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:46.174391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ea2f0-7b0b-466a-9aff-40ef0cab8e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:46.185784 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ea2f0-7b0b-466a-9aff-40ef0cab8e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:46.187225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ea2f0-7b0b-466a-9aff-40ef0cab8e34 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:46.197406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1396/2753] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:00:46.456396 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a1f07-5c35-4ff7-8393-a8865b6b8657 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:00:46.457927 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ea0a1f07-5c35-4ff7-8393-a8865b6b8657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:00:46.471674 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a1f07-5c35-4ff7-8393-a8865b6b8657 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:00:46.477125 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:00:46.477818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:00:46.480888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:00:46.480888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] PATH_INFO: `/v3/projects/84398f99ba3e4244b06ee667c05c5d52` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:00:46.480888 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84398f99ba3e4244b06ee667c05c5d52)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:00:46.481470 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:00:46.612696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:46.618271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84398f99ba3e4244b06ee667c05c5d52', 'request_id': 'req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:46.622931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84398f99ba3e4244b06ee667c05c5d52', 'request_id': 'req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:46.628580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84398f99ba3e4244b06ee667c05c5d52', 'request_id': 'req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:46.699315 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] The token cache is being invalidate because project 84398f99ba3e4244b06ee667c05c5d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:00:46.699534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:46.700803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:00:46.702390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1358/2754] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:00:46 2022] DELETE /identity/v3/projects/84398f99ba3e4244b06ee667c05c5d52 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 19:00:56.192832 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=29,keystone:INSERT=1,keystone:DELETE=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:00:56.705456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:01:14.136063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad66d-0b28-420a-b0d6-f693a04c54e8 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:01:14.136580 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad66d-0b28-420a-b0d6-f693a04c54e8 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:01:14.136702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad66d-0b28-420a-b0d6-f693a04c54e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:01:14.156311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-5efad66d-0b28-420a-b0d6-f693a04c54e8 None None] MFA Rules not processed for user `1a5c2d856c8f4bfc9d59b976ca4371ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:01:14.157552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5efad66d-0b28-420a-b0d6-f693a04c54e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:01:14.211956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1397/2755] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:01:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:01:24.015393 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:01:24.016885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:01:24.073165 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307dbbc-0edd-4973-b3a5-1bbafb79dc20 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:01:24.076678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0a0c8-a3b4-4ddb-9a8b-7d68ff055bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:01:24.077297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0a0c8-a3b4-4ddb-9a8b-7d68ff055bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:01:24.077577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0a0c8-a3b4-4ddb-9a8b-7d68ff055bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:01:24.077847 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0a0c8-a3b4-4ddb-9a8b-7d68ff055bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:01:24.079633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-74d0a0c8-a3b4-4ddb-9a8b-7d68ff055bb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:01:24.161260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0a0c8-a3b4-4ddb-9a8b-7d68ff055bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:01:24.162681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0a0c8-a3b4-4ddb-9a8b-7d68ff055bb2 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:01:24.183064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1359/2756] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:01:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:01:24.204755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=25 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:01:26.761755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad66d-0b28-420a-b0d6-f693a04c54e8 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:01:26.763243 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-5efad66d-0b28-420a-b0d6-f693a04c54e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:01:26.822892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad66d-0b28-420a-b0d6-f693a04c54e8 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:01:26.826516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14fe8a-87fe-473f-9d84-9cd5e62ab703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:01:26.827119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14fe8a-87fe-473f-9d84-9cd5e62ab703 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:01:26.827462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14fe8a-87fe-473f-9d84-9cd5e62ab703 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:01:26.827741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14fe8a-87fe-473f-9d84-9cd5e62ab703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:01:26.829489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8a14fe8a-87fe-473f-9d84-9cd5e62ab703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:01:26.845203 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14fe8a-87fe-473f-9d84-9cd5e62ab703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:01:26.846690 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14fe8a-87fe-473f-9d84-9cd5e62ab703 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:01:26.857820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1398/2757] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:01:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:01:34.180560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=63 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:01:36.853007 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=26 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:02:07.457997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0a0c8-a3b4-4ddb-9a8b-7d68ff055bb2 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:02:07.459462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-74d0a0c8-a3b4-4ddb-9a8b-7d68ff055bb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:02:07.470913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0a0c8-a3b4-4ddb-9a8b-7d68ff055bb2 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:02:07.475694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee3a36-f753-47ed-b992-8cc56fb16a88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:02:07.476295 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee3a36-f753-47ed-b992-8cc56fb16a88 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:02:07.476577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee3a36-f753-47ed-b992-8cc56fb16a88 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:02:07.476855 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee3a36-f753-47ed-b992-8cc56fb16a88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:02:07.478521 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f7ee3a36-f753-47ed-b992-8cc56fb16a88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:02:07.541117 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee3a36-f753-47ed-b992-8cc56fb16a88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:02:07.543065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee3a36-f753-47ed-b992-8cc56fb16a88 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:02:07.552937 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1360/2758] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:02:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:02:17.548551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=28 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:02:25.034636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14fe8a-87fe-473f-9d84-9cd5e62ab703 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:02:25.036757 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8a14fe8a-87fe-473f-9d84-9cd5e62ab703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:02:25.048192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14fe8a-87fe-473f-9d84-9cd5e62ab703 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:02:25.051980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cce23-9997-406b-9241-3425ea305b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:02:25.052583 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cce23-9997-406b-9241-3425ea305b20 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:02:25.052860 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cce23-9997-406b-9241-3425ea305b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:02:25.053164 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cce23-9997-406b-9241-3425ea305b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:02:25.060367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f21cce23-9997-406b-9241-3425ea305b20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:02:25.064845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cce23-9997-406b-9241-3425ea305b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:02:25.067053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cce23-9997-406b-9241-3425ea305b20 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:02:25.075674 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1399/2759] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:02:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:02:35.071902 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:04:59.372961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee3a36-f753-47ed-b992-8cc56fb16a88 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:04:59.381864 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee3a36-f753-47ed-b992-8cc56fb16a88 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:04:59.386975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1b5b7-533a-4339-b3ca-c9ec93c07aef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:04:59.387856 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1b5b7-533a-4339-b3ca-c9ec93c07aef service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:04:59.388272 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1b5b7-533a-4339-b3ca-c9ec93c07aef service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:04:59.388678 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1b5b7-533a-4339-b3ca-c9ec93c07aef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:04:59.391096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-62f1b5b7-533a-4339-b3ca-c9ec93c07aef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:04:59.406584 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1b5b7-533a-4339-b3ca-c9ec93c07aef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:04:59.408143 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1b5b7-533a-4339-b3ca-c9ec93c07aef service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:04:59.421038 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1361/2760] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:04:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:05:09.413188 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:05:41.232213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cce23-9997-406b-9241-3425ea305b20 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:05:41.233626 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f21cce23-9997-406b-9241-3425ea305b20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:05:41.303443 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cce23-9997-406b-9241-3425ea305b20 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:05:41.308870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ddc64-f64f-4890-bc7f-b465e4b0d97e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:05:41.313552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ddc64-f64f-4890-bc7f-b465e4b0d97e service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:05:41.313842 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ddc64-f64f-4890-bc7f-b465e4b0d97e service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:05:41.314122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ddc64-f64f-4890-bc7f-b465e4b0d97e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:05:41.315950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6b6ddc64-f64f-4890-bc7f-b465e4b0d97e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:05:41.329001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ddc64-f64f-4890-bc7f-b465e4b0d97e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:05:41.330619 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ddc64-f64f-4890-bc7f-b465e4b0d97e service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:05:41.341707 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1400/2761] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:05:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:05:51.337187 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=32 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:07:07.794027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1b5b7-533a-4339-b3ca-c9ec93c07aef service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:07:07.802848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1b5b7-533a-4339-b3ca-c9ec93c07aef service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:07:07.806731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec0a21-20b2-4df5-94d1-5145c20e4ca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:07:07.807361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec0a21-20b2-4df5-94d1-5145c20e4ca7 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:07:07.807639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec0a21-20b2-4df5-94d1-5145c20e4ca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:07:07.807911 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec0a21-20b2-4df5-94d1-5145c20e4ca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:07:07.816294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec0a21-20b2-4df5-94d1-5145c20e4ca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:07:07.817757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec0a21-20b2-4df5-94d1-5145c20e4ca7 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:07:07.845749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1362/2762] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:07:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:07:17.842961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=13 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:07:40.012567 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ddc64-f64f-4890-bc7f-b465e4b0d97e service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:07:40.021967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ddc64-f64f-4890-bc7f-b465e4b0d97e service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:07:40.026454 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d7915-a1bd-4472-9449-04b5b55c6eea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:07:40.027041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d7915-a1bd-4472-9449-04b5b55c6eea service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:07:40.027316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d7915-a1bd-4472-9449-04b5b55c6eea service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:07:40.027587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d7915-a1bd-4472-9449-04b5b55c6eea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:07:40.037782 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d7915-a1bd-4472-9449-04b5b55c6eea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:07:40.039235 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d7915-a1bd-4472-9449-04b5b55c6eea service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:07:40.050090 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1401/2763] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:07:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:07:45.921343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec0a21-20b2-4df5-94d1-5145c20e4ca7 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:07:45.935765 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec0a21-20b2-4df5-94d1-5145c20e4ca7 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:07:45.940007 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40129827-ab22-41ad-bf35-3b4ec0d4c5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:07:45.940657 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40129827-ab22-41ad-bf35-3b4ec0d4c5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:07:45.940934 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40129827-ab22-41ad-bf35-3b4ec0d4c5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:07:45.941232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40129827-ab22-41ad-bf35-3b4ec0d4c5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:07:45.949644 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40129827-ab22-41ad-bf35-3b4ec0d4c5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:07:45.951083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40129827-ab22-41ad-bf35-3b4ec0d4c5e8 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:07:45.959440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1363/2764] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:07:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 19:07:50.045147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:07:55.955840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:08:29.233915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d7915-a1bd-4472-9449-04b5b55c6eea service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:08:29.243184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d7915-a1bd-4472-9449-04b5b55c6eea service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:08:29.246803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db0708-8a43-4163-8dc4-a1bd3393e840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:08:29.247389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db0708-8a43-4163-8dc4-a1bd3393e840 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:08:29.247663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db0708-8a43-4163-8dc4-a1bd3393e840 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:08:29.248055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db0708-8a43-4163-8dc4-a1bd3393e840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:08:29.259194 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db0708-8a43-4163-8dc4-a1bd3393e840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:08:29.260639 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db0708-8a43-4163-8dc4-a1bd3393e840 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:08:29.271385 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1402/2765] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:08:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:08:39.266600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:09:59.044844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40129827-ab22-41ad-bf35-3b4ec0d4c5e8 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:09:59.081724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40129827-ab22-41ad-bf35-3b4ec0d4c5e8 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:09:59.085320 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48118e65-11e1-40c0-bdfe-598ccdb5e722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:09:59.085909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48118e65-11e1-40c0-bdfe-598ccdb5e722 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:09:59.086181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48118e65-11e1-40c0-bdfe-598ccdb5e722 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:09:59.086447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48118e65-11e1-40c0-bdfe-598ccdb5e722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:09:59.107452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48118e65-11e1-40c0-bdfe-598ccdb5e722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:09:59.108871 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48118e65-11e1-40c0-bdfe-598ccdb5e722 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:09:59.117878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1364/2766] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:09:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:10:09.113754 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=24 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:10:43.234135 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db0708-8a43-4163-8dc4-a1bd3393e840 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:10:43.287011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db0708-8a43-4163-8dc4-a1bd3393e840 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:10:43.292184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f7b76-c7e8-432c-9386-a5d7a05daff4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:10:43.292780 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f7b76-c7e8-432c-9386-a5d7a05daff4 service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:10:43.293065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f7b76-c7e8-432c-9386-a5d7a05daff4 service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:10:43.293359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f7b76-c7e8-432c-9386-a5d7a05daff4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:10:43.301683 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f7b76-c7e8-432c-9386-a5d7a05daff4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:10:43.303236 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f7b76-c7e8-432c-9386-a5d7a05daff4 service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:10:43.313655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1403/2767] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:10:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:10:53.309411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=24 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:12:07.952929 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48118e65-11e1-40c0-bdfe-598ccdb5e722 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:12:07.954342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-48118e65-11e1-40c0-bdfe-598ccdb5e722 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:12:08.065797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48118e65-11e1-40c0-bdfe-598ccdb5e722 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:12:08.071325 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dc4a0-ce3d-4699-b636-ceb0230fb9d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:12:08.072434 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dc4a0-ce3d-4699-b636-ceb0230fb9d9 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:12:08.072723 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dc4a0-ce3d-4699-b636-ceb0230fb9d9 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:12:08.072996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dc4a0-ce3d-4699-b636-ceb0230fb9d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:12:08.074951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2c9dc4a0-ce3d-4699-b636-ceb0230fb9d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:12:08.143407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dc4a0-ce3d-4699-b636-ceb0230fb9d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:12:08.144982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dc4a0-ce3d-4699-b636-ceb0230fb9d9 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:12:08.154918 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1365/2768] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:12:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:12:18.149829 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=77 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:12:50.917534 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f7b76-c7e8-432c-9386-a5d7a05daff4 service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:12:50.919118 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-795f7b76-c7e8-432c-9386-a5d7a05daff4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:12:50.995889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f7b76-c7e8-432c-9386-a5d7a05daff4 service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:12:50.999460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20866-d928-4d10-b741-aea3f2c34638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:12:51.000055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20866-d928-4d10-b741-aea3f2c34638 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:12:51.000332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20866-d928-4d10-b741-aea3f2c34638 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:12:51.000602 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac20866-d928-4d10-b741-aea3f2c34638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:12:51.002418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-7ac20866-d928-4d10-b741-aea3f2c34638 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:12:51.012212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac20866-d928-4d10-b741-aea3f2c34638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:12:51.013652 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac20866-d928-4d10-b741-aea3f2c34638 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:12:51.021601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1404/2769] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:12:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:13:01.017961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=35 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:13:40.827513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dc4a0-ce3d-4699-b636-ceb0230fb9d9 service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:13:40.829178 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2c9dc4a0-ce3d-4699-b636-ceb0230fb9d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:13:40.841221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dc4a0-ce3d-4699-b636-ceb0230fb9d9 service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:13:40.844694 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475bf6f-e316-4551-94a1-9791fc60e059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:13:40.845318 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475bf6f-e316-4551-94a1-9791fc60e059 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:13:40.845595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475bf6f-e316-4551-94a1-9791fc60e059 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:13:40.845869 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475bf6f-e316-4551-94a1-9791fc60e059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:13:40.847616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3475bf6f-e316-4551-94a1-9791fc60e059 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:13:40.861322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475bf6f-e316-4551-94a1-9791fc60e059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:13:40.862735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475bf6f-e316-4551-94a1-9791fc60e059 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:13:40.874310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1366/2770] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:13:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:13:50.869195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:14:31.424391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20866-d928-4d10-b741-aea3f2c34638 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:14:31.436183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac20866-d928-4d10-b741-aea3f2c34638 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:14:31.439803 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851df452-fb11-488a-9420-be7a0459a04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:14:31.440401 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851df452-fb11-488a-9420-be7a0459a04a service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:14:31.440677 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851df452-fb11-488a-9420-be7a0459a04a service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:14:31.440947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851df452-fb11-488a-9420-be7a0459a04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:14:31.442792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-851df452-fb11-488a-9420-be7a0459a04a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:14:31.456880 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851df452-fb11-488a-9420-be7a0459a04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:14:31.458324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851df452-fb11-488a-9420-be7a0459a04a service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:14:31.470436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1405/2771] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:14:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:14:41.465543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:14:59.858574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475bf6f-e316-4551-94a1-9791fc60e059 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:14:59.870242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475bf6f-e316-4551-94a1-9791fc60e059 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:14:59.875974 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c2287-70c1-4951-b372-c1628af29139 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:14:59.876578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c2287-70c1-4951-b372-c1628af29139 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:14:59.876859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c2287-70c1-4951-b372-c1628af29139 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:14:59.877187 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c2287-70c1-4951-b372-c1628af29139 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:14:59.879004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9b1c2287-70c1-4951-b372-c1628af29139 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:14:59.958354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c2287-70c1-4951-b372-c1628af29139 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:14:59.960152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c2287-70c1-4951-b372-c1628af29139 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:14:59.980653 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1367/2772] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:14:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:15:09.977690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=41 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:15:49.233009 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851df452-fb11-488a-9420-be7a0459a04a service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:15:49.234582 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-851df452-fb11-488a-9420-be7a0459a04a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:15:49.315930 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851df452-fb11-488a-9420-be7a0459a04a service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:15:49.321826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece217e1-0301-405b-882b-4f9ef15f0516 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:15:49.322447 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece217e1-0301-405b-882b-4f9ef15f0516 service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:15:49.322727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece217e1-0301-405b-882b-4f9ef15f0516 service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:15:49.323000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece217e1-0301-405b-882b-4f9ef15f0516 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:15:49.324911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ece217e1-0301-405b-882b-4f9ef15f0516 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:15:49.337245 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece217e1-0301-405b-882b-4f9ef15f0516 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:15:49.338830 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece217e1-0301-405b-882b-4f9ef15f0516 service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:15:49.349129 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1406/2773] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:15:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:15:59.344634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=37 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:17:06.629372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c2287-70c1-4951-b372-c1628af29139 service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:17:06.632211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9b1c2287-70c1-4951-b372-c1628af29139 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:17:06.702093 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c2287-70c1-4951-b372-c1628af29139 service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:17:06.705805 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634681e3-2f7f-44fe-9cd4-26fb828e0e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:17:06.706409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634681e3-2f7f-44fe-9cd4-26fb828e0e18 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:17:06.706719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634681e3-2f7f-44fe-9cd4-26fb828e0e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:17:06.707044 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634681e3-2f7f-44fe-9cd4-26fb828e0e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:17:06.715758 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634681e3-2f7f-44fe-9cd4-26fb828e0e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:17:06.717218 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634681e3-2f7f-44fe-9cd4-26fb828e0e18 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:17:06.726155 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1368/2774] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:17:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:17:16.721973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=31 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:17:43.561719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece217e1-0301-405b-882b-4f9ef15f0516 service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:17:43.577730 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece217e1-0301-405b-882b-4f9ef15f0516 service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:17:43.583253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0973b-dcae-4acb-beea-2648e2f441a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:17:43.583852 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0973b-dcae-4acb-beea-2648e2f441a3 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:17:43.584132 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0973b-dcae-4acb-beea-2648e2f441a3 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:17:43.584408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0973b-dcae-4acb-beea-2648e2f441a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:17:43.586197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8fc0973b-dcae-4acb-beea-2648e2f441a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:17:43.599380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0973b-dcae-4acb-beea-2648e2f441a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:17:43.601012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0973b-dcae-4acb-beea-2648e2f441a3 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:17:43.612242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1407/2775] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:17:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:17:53.674446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:17:53.797060 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634681e3-2f7f-44fe-9cd4-26fb828e0e18 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:17:53.807443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634681e3-2f7f-44fe-9cd4-26fb828e0e18 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:17:53.810928 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f966384-277a-40ee-bd0c-9051f66d8640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:17:53.811514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966384-277a-40ee-bd0c-9051f66d8640 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:17:53.811790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966384-277a-40ee-bd0c-9051f66d8640 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:17:53.812061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f966384-277a-40ee-bd0c-9051f66d8640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:17:53.819378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f966384-277a-40ee-bd0c-9051f66d8640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:17:53.821773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f966384-277a-40ee-bd0c-9051f66d8640 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:17:53.830042 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1369/2776] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:17:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:18:03.826291 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:18:42.190702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0973b-dcae-4acb-beea-2648e2f441a3 service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:18:42.200682 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0973b-dcae-4acb-beea-2648e2f441a3 service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:18:42.204216 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb751a88-fa0e-4f8a-958a-844349a74022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:18:42.204840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb751a88-fa0e-4f8a-958a-844349a74022 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:18:42.205157 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb751a88-fa0e-4f8a-958a-844349a74022 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:18:42.205432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb751a88-fa0e-4f8a-958a-844349a74022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:18:42.215608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb751a88-fa0e-4f8a-958a-844349a74022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:18:42.217012 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb751a88-fa0e-4f8a-958a-844349a74022 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:18:42.227900 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1408/2777] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:18:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:18:52.222924 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:19:32.229631 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f966384-277a-40ee-bd0c-9051f66d8640 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:19:32.238390 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f966384-277a-40ee-bd0c-9051f66d8640 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:19:32.242078 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3659789-2941-4d36-9ef0-e356c5c3af13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:19:32.242672 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3659789-2941-4d36-9ef0-e356c5c3af13 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:19:32.242957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3659789-2941-4d36-9ef0-e356c5c3af13 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:19:32.243853 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3659789-2941-4d36-9ef0-e356c5c3af13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:19:32.254234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3659789-2941-4d36-9ef0-e356c5c3af13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:19:32.255649 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3659789-2941-4d36-9ef0-e356c5c3af13 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:19:32.266881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1370/2778] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:19:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:19:42.261766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:20:56.208147 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb751a88-fa0e-4f8a-958a-844349a74022 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:20:56.268202 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb751a88-fa0e-4f8a-958a-844349a74022 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:20:56.273551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca57ae6-b8d9-4f1f-8796-a867ed0a72de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:20:56.274153 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57ae6-b8d9-4f1f-8796-a867ed0a72de service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:20:56.274433 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57ae6-b8d9-4f1f-8796-a867ed0a72de service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:20:56.274709 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57ae6-b8d9-4f1f-8796-a867ed0a72de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:20:56.284186 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca57ae6-b8d9-4f1f-8796-a867ed0a72de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:20:56.285787 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca57ae6-b8d9-4f1f-8796-a867ed0a72de service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:20:56.295715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1409/2779] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:20:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:21:06.291416 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=27 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:21:42.863123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3659789-2941-4d36-9ef0-e356c5c3af13 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:21:42.864648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a3659789-2941-4d36-9ef0-e356c5c3af13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:21:42.896921 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3659789-2941-4d36-9ef0-e356c5c3af13 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:21:42.900549 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56266381-e017-4827-9c3f-a65f4ae2abaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:21:42.901181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56266381-e017-4827-9c3f-a65f4ae2abaf service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:21:42.901490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56266381-e017-4827-9c3f-a65f4ae2abaf service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:21:42.901760 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56266381-e017-4827-9c3f-a65f4ae2abaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:21:42.910289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56266381-e017-4827-9c3f-a65f4ae2abaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:21:42.911700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56266381-e017-4827-9c3f-a65f4ae2abaf service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:21:42.920461 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1371/2780] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:21:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:21:52.916474 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=15 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:22:06.943377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca57ae6-b8d9-4f1f-8796-a867ed0a72de service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:22:06.944856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-eca57ae6-b8d9-4f1f-8796-a867ed0a72de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:22:06.961861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca57ae6-b8d9-4f1f-8796-a867ed0a72de service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:22:06.967282 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81e005-25fc-49bf-b75b-fc29f16f2d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:22:06.967901 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81e005-25fc-49bf-b75b-fc29f16f2d58 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:22:06.968184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81e005-25fc-49bf-b75b-fc29f16f2d58 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:22:06.968459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81e005-25fc-49bf-b75b-fc29f16f2d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:22:06.977477 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81e005-25fc-49bf-b75b-fc29f16f2d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:22:06.979033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81e005-25fc-49bf-b75b-fc29f16f2d58 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:22:07.007426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1410/2781] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:22:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:22:17.004838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=13 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:23:43.735810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56266381-e017-4827-9c3f-a65f4ae2abaf service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:23:43.737448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-56266381-e017-4827-9c3f-a65f4ae2abaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:23:43.826402 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56266381-e017-4827-9c3f-a65f4ae2abaf service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:23:43.830072 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80627f99-556d-4401-8243-3488b62fc6d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:23:43.830667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80627f99-556d-4401-8243-3488b62fc6d2 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:23:43.830947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80627f99-556d-4401-8243-3488b62fc6d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:23:43.831223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80627f99-556d-4401-8243-3488b62fc6d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:23:43.833022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-80627f99-556d-4401-8243-3488b62fc6d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:23:43.922035 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80627f99-556d-4401-8243-3488b62fc6d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:23:43.923795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80627f99-556d-4401-8243-3488b62fc6d2 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:23:43.944895 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1372/2782] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:23:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:23:53.942306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=79 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:24:32.320402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81e005-25fc-49bf-b75b-fc29f16f2d58 service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:24:32.330521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81e005-25fc-49bf-b75b-fc29f16f2d58 service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:24:32.334123 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fda842-45dd-4b0e-836b-d4f3ad3bd9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:24:32.334714 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fda842-45dd-4b0e-836b-d4f3ad3bd9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:24:32.334992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fda842-45dd-4b0e-836b-d4f3ad3bd9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:24:32.335262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fda842-45dd-4b0e-836b-d4f3ad3bd9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:24:32.337087 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-00fda842-45dd-4b0e-836b-d4f3ad3bd9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:24:32.350820 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fda842-45dd-4b0e-836b-d4f3ad3bd9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:24:32.352233 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fda842-45dd-4b0e-836b-d4f3ad3bd9e4 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:24:32.362991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1411/2783] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:24:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:24:42.358175 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:26:03.199857 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80627f99-556d-4401-8243-3488b62fc6d2 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:26:03.201448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-80627f99-556d-4401-8243-3488b62fc6d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:26:03.281305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80627f99-556d-4401-8243-3488b62fc6d2 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:26:03.287079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f410333-1ab7-44ff-8be6-53d26d4a043b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:26:03.287700 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f410333-1ab7-44ff-8be6-53d26d4a043b service placement] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:26:03.287975 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f410333-1ab7-44ff-8be6-53d26d4a043b service placement] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:26:03.288250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f410333-1ab7-44ff-8be6-53d26d4a043b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:26:03.290068 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2f410333-1ab7-44ff-8be6-53d26d4a043b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:26:03.301344 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f410333-1ab7-44ff-8be6-53d26d4a043b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:26:03.303097 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f410333-1ab7-44ff-8be6-53d26d4a043b service placement] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:26:03.312733 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1373/2784] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:26:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:26:13.308622 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=37 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:27:07.563017 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fda842-45dd-4b0e-836b-d4f3ad3bd9e4 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:27:07.564680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-00fda842-45dd-4b0e-836b-d4f3ad3bd9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:27:07.578302 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fda842-45dd-4b0e-836b-d4f3ad3bd9e4 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:27:07.584200 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43559f-dd5e-489b-90b0-85787c147332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:27:07.584785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43559f-dd5e-489b-90b0-85787c147332 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:27:07.585085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43559f-dd5e-489b-90b0-85787c147332 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:27:07.585364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43559f-dd5e-489b-90b0-85787c147332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:27:07.587110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-af43559f-dd5e-489b-90b0-85787c147332 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:27:07.663710 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43559f-dd5e-489b-90b0-85787c147332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:27:07.665318 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43559f-dd5e-489b-90b0-85787c147332 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:27:07.678566 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1412/2785] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:27:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:27:17.670873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=33 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:27:43.305909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f410333-1ab7-44ff-8be6-53d26d4a043b service placement] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:27:43.365920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f410333-1ab7-44ff-8be6-53d26d4a043b service placement] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:27:43.369520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ccd0c-3e60-4d72-91ca-0f76e42b5a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:27:43.370196 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ccd0c-3e60-4d72-91ca-0f76e42b5a64 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:27:43.370534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ccd0c-3e60-4d72-91ca-0f76e42b5a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:27:43.370811 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ccd0c-3e60-4d72-91ca-0f76e42b5a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:27:43.372686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-704ccd0c-3e60-4d72-91ca-0f76e42b5a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:27:43.473238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ccd0c-3e60-4d72-91ca-0f76e42b5a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:27:43.475013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ccd0c-3e60-4d72-91ca-0f76e42b5a64 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:27:43.494617 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1374/2786] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:27:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:27:44.394634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43559f-dd5e-489b-90b0-85787c147332 service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:27:44.407695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43559f-dd5e-489b-90b0-85787c147332 service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:27:44.415136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81961c0d-5bd1-4ee8-9fb3-aba2160c7e29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:27:44.416048 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81961c0d-5bd1-4ee8-9fb3-aba2160c7e29 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:27:44.416575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81961c0d-5bd1-4ee8-9fb3-aba2160c7e29 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:27:44.417074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81961c0d-5bd1-4ee8-9fb3-aba2160c7e29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:27:44.420191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-81961c0d-5bd1-4ee8-9fb3-aba2160c7e29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:27:44.436622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81961c0d-5bd1-4ee8-9fb3-aba2160c7e29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:27:44.438513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81961c0d-5bd1-4ee8-9fb3-aba2160c7e29 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:27:44.450853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1413/2787] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:27:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:27:44.558834 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ccd0c-3e60-4d72-91ca-0f76e42b5a64 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:27:44.572182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ccd0c-3e60-4d72-91ca-0f76e42b5a64 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:27:44.578148 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ceaf2-87e7-4b2a-8430-69d45e31e4af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:27:44.578998 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ceaf2-87e7-4b2a-8430-69d45e31e4af service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:27:44.579469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ceaf2-87e7-4b2a-8430-69d45e31e4af service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:27:44.579977 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ceaf2-87e7-4b2a-8430-69d45e31e4af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:27:44.582446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ec1ceaf2-87e7-4b2a-8430-69d45e31e4af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:27:44.611413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ceaf2-87e7-4b2a-8430-69d45e31e4af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:27:44.613101 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ceaf2-87e7-4b2a-8430-69d45e31e4af service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:27:44.666214 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1375/2788] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:27:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:27:44.932991 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81961c0d-5bd1-4ee8-9fb3-aba2160c7e29 service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:27:44.945979 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81961c0d-5bd1-4ee8-9fb3-aba2160c7e29 service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:27:44.953805 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f05c3-a2af-4bc7-bcec-ff74fbf16120 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:27:44.961426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f05c3-a2af-4bc7-bcec-ff74fbf16120 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:27:44.961859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f05c3-a2af-4bc7-bcec-ff74fbf16120 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:27:44.962144 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f05c3-a2af-4bc7-bcec-ff74fbf16120 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:27:44.964266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6b5f05c3-a2af-4bc7-bcec-ff74fbf16120 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:27:44.977577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f05c3-a2af-4bc7-bcec-ff74fbf16120 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:27:44.979204 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f05c3-a2af-4bc7-bcec-ff74fbf16120 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:27:44.993909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1414/2789] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:27:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:27:54.636376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=67 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:27:54.985944 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=12 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:31:07.176418 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ceaf2-87e7-4b2a-8430-69d45e31e4af service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:31:07.177828 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ec1ceaf2-87e7-4b2a-8430-69d45e31e4af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:31:07.233763 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ceaf2-87e7-4b2a-8430-69d45e31e4af service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:31:07.238826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaf733-55d9-4cc2-b715-dc36447c8503 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:31:07.239423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaf733-55d9-4cc2-b715-dc36447c8503 service placement] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:31:07.239701 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaf733-55d9-4cc2-b715-dc36447c8503 service placement] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:31:07.239973 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaaf733-55d9-4cc2-b715-dc36447c8503 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:31:07.248654 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaaf733-55d9-4cc2-b715-dc36447c8503 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:31:07.250273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaaf733-55d9-4cc2-b715-dc36447c8503 service placement] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:31:07.260189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1376/2790] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:31:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:31:17.255789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=23 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:32:08.206073 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f05c3-a2af-4bc7-bcec-ff74fbf16120 service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:08.207656 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6b5f05c3-a2af-4bc7-bcec-ff74fbf16120 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:08.263514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f05c3-a2af-4bc7-bcec-ff74fbf16120 service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:08.267168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c0e54-f854-4854-b3fc-a1ac3b417997 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:08.267770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c0e54-f854-4854-b3fc-a1ac3b417997 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:08.268052 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c0e54-f854-4854-b3fc-a1ac3b417997 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:08.268327 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c0e54-f854-4854-b3fc-a1ac3b417997 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:08.276966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c0e54-f854-4854-b3fc-a1ac3b417997 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:08.278681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c0e54-f854-4854-b3fc-a1ac3b417997 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:08.305444 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1415/2791] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:32:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:32:18.302875 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=31 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:32:44.096502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaf733-55d9-4cc2-b715-dc36447c8503 service placement] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:44.098135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-efaaf733-55d9-4cc2-b715-dc36447c8503 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:44.123618 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaaf733-55d9-4cc2-b715-dc36447c8503 service placement] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:44.128049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e189516-2c73-4556-8520-98a6cb96f804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:44.128655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e189516-2c73-4556-8520-98a6cb96f804 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:44.128936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e189516-2c73-4556-8520-98a6cb96f804 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:44.129243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e189516-2c73-4556-8520-98a6cb96f804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:44.137539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e189516-2c73-4556-8520-98a6cb96f804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:44.138965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e189516-2c73-4556-8520-98a6cb96f804 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:44.148759 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1377/2792] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:32:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:32:45.546761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c0e54-f854-4854-b3fc-a1ac3b417997 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:45.556768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c0e54-f854-4854-b3fc-a1ac3b417997 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:45.560305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ad77a-8d7c-40ec-b238-9c362b0a1aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:45.560893 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad77a-8d7c-40ec-b238-9c362b0a1aab service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:45.561214 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad77a-8d7c-40ec-b238-9c362b0a1aab service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:45.561484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad77a-8d7c-40ec-b238-9c362b0a1aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:45.563721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b34ad77a-8d7c-40ec-b238-9c362b0a1aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:45.650938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ad77a-8d7c-40ec-b238-9c362b0a1aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:45.652380 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ad77a-8d7c-40ec-b238-9c362b0a1aab service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:45.671825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1416/2793] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:32:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:32:46.194208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e189516-2c73-4556-8520-98a6cb96f804 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:46.206986 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e189516-2c73-4556-8520-98a6cb96f804 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:46.212184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a1435-3c29-427e-9062-c9c3c8bfd938 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:46.212787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a1435-3c29-427e-9062-c9c3c8bfd938 service placement] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:46.213107 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a1435-3c29-427e-9062-c9c3c8bfd938 service placement] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:46.213379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a1435-3c29-427e-9062-c9c3c8bfd938 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:46.215303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-a37a1435-3c29-427e-9062-c9c3c8bfd938 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:46.227884 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a1435-3c29-427e-9062-c9c3c8bfd938 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:46.229503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a1435-3c29-427e-9062-c9c3c8bfd938 service placement] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:46.239755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1378/2794] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:32:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:32:46.314015 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ad77a-8d7c-40ec-b238-9c362b0a1aab service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:46.315584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b34ad77a-8d7c-40ec-b238-9c362b0a1aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:46.327621 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ad77a-8d7c-40ec-b238-9c362b0a1aab service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:46.333096 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:46.333692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:46.333869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:46.334104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] PATH_INFO: `/v3/users/1a5c2d856c8f4bfc9d59b976ca4371ee` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:46.335213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a5c2d856c8f4bfc9d59b976ca4371ee)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:46.337152 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:46.503732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a5c2d856c8f4bfc9d59b976ca4371ee'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:32:46.504397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a5c2d856c8f4bfc9d59b976ca4371ee'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:32:46.505014 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a5c2d856c8f4bfc9d59b976ca4371ee'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:32:46.512184 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a5c2d856c8f4bfc9d59b976ca4371ee'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:32:46.525390 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-9b42219e-031b-4ee9-afd6-03261c6f80f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a5c2d856c8f4bfc9d59b976ca4371ee'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:32:46.615174 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1417/2795] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:32:46 2022] DELETE /identity/v3/users/1a5c2d856c8f4bfc9d59b976ca4371ee => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:32:46.868623 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a1435-3c29-427e-9062-c9c3c8bfd938 service placement] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:46.880307 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a1435-3c29-427e-9062-c9c3c8bfd938 service placement] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:46.940818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:46.941482 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:46.941766 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:46.942042 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] PATH_INFO: `/v3/projects/1235f57770f34c87aeb3369fa8e04b54` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:46.943193 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1235f57770f34c87aeb3369fa8e04b54)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:46.945173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:47.050796 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1235f57770f34c87aeb3369fa8e04b54', 'request_id': 'req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:32:47.057010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1235f57770f34c87aeb3369fa8e04b54', 'request_id': 'req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:32:47.063552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1235f57770f34c87aeb3369fa8e04b54', 'request_id': 'req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:32:47.140052 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] The token cache is being invalidate because project 1235f57770f34c87aeb3369fa8e04b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:32:47.140262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:32:47.141471 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-94a7b33b-bad2-40c8-8eb5-805c194f6b8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:32:47.143046 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1379/2796] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:32:46 2022] DELETE /identity/v3/projects/1235f57770f34c87aeb3369fa8e04b54 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:32:47.148872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab2f62-4223-4234-bde6-6f47668dacb9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:47.149197 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab2f62-4223-4234-bde6-6f47668dacb9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:47.149487 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab2f62-4223-4234-bde6-6f47668dacb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:47.205644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:32:47.208490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-d2ab2f62-4223-4234-bde6-6f47668dacb9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:32:47.209740 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d2ab2f62-4223-4234-bde6-6f47668dacb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:47.269036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1418/2797] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:32:47.277199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9e686-6bcf-4b6f-81f7-70e31fd16781 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:47.278556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9e686-6bcf-4b6f-81f7-70e31fd16781 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:47.278556 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9e686-6bcf-4b6f-81f7-70e31fd16781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:47.298931 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-5ff9e686-6bcf-4b6f-81f7-70e31fd16781 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:32:47.300138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5ff9e686-6bcf-4b6f-81f7-70e31fd16781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:47.308136 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1380/2798] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:32:47.312062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab2f62-4223-4234-bde6-6f47668dacb9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:47.322513 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab2f62-4223-4234-bde6-6f47668dacb9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:47.327136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2aaca1-669b-4a7e-96f4-05c4562adfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:47.327737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2aaca1-669b-4a7e-96f4-05c4562adfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:47.328019 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2aaca1-669b-4a7e-96f4-05c4562adfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:47.328878 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2aaca1-669b-4a7e-96f4-05c4562adfd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:47.329565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2aaca1-669b-4a7e-96f4-05c4562adfd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 19:32:47.329874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2aaca1-669b-4a7e-96f4-05c4562adfd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:47.331468 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2aaca1-669b-4a7e-96f4-05c4562adfd7 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:47.343431 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1419/2799] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 19:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 19:32:47.348768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9e686-6bcf-4b6f-81f7-70e31fd16781 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:47.360546 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9e686-6bcf-4b6f-81f7-70e31fd16781 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:47.365267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0c33a-28cc-4503-a7e0-a73c65dc3248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:47.365862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c33a-28cc-4503-a7e0-a73c65dc3248 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:47.366142 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c33a-28cc-4503-a7e0-a73c65dc3248 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:47.366421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c33a-28cc-4503-a7e0-a73c65dc3248 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:47.367044 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0c33a-28cc-4503-a7e0-a73c65dc3248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:47.368930 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0c33a-28cc-4503-a7e0-a73c65dc3248 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:47.493985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1381/2800] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:32:47 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:32:47.498136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2aaca1-669b-4a7e-96f4-05c4562adfd7 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:47.509093 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2aaca1-669b-4a7e-96f4-05c4562adfd7 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:47.566702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da3e65-c726-447e-bc05-86abf07da539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:47.567305 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da3e65-c726-447e-bc05-86abf07da539 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:47.567589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da3e65-c726-447e-bc05-86abf07da539 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:47.567864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da3e65-c726-447e-bc05-86abf07da539 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:47.568500 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da3e65-c726-447e-bc05-86abf07da539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:47.570437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da3e65-c726-447e-bc05-86abf07da539 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:47.683850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1420/2801] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:32:47 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:32:47.688827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0c33a-28cc-4503-a7e0-a73c65dc3248 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:47.699457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0c33a-28cc-4503-a7e0-a73c65dc3248 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:47.704064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab43a85-ce8d-4c16-9503-355fb2ff3f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:47.704728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43a85-ce8d-4c16-9503-355fb2ff3f4c admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:47.705003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43a85-ce8d-4c16-9503-355fb2ff3f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:47.705306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43a85-ce8d-4c16-9503-355fb2ff3f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:47.705948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab43a85-ce8d-4c16-9503-355fb2ff3f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:47.707525 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab43a85-ce8d-4c16-9503-355fb2ff3f4c admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:47.717462 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1382/2802] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:32:47.721565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da3e65-c726-447e-bc05-86abf07da539 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:47.731556 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da3e65-c726-447e-bc05-86abf07da539 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:47.736966 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f88bc5-c740-4722-8c81-2e3c3a0a1c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:47.737593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f88bc5-c740-4722-8c81-2e3c3a0a1c01 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:47.737870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f88bc5-c740-4722-8c81-2e3c3a0a1c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:47.738141 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f88bc5-c740-4722-8c81-2e3c3a0a1c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:47.738779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f88bc5-c740-4722-8c81-2e3c3a0a1c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:47.740351 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f88bc5-c740-4722-8c81-2e3c3a0a1c01 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:47.752061 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1421/2803] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:32:47.756268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab43a85-ce8d-4c16-9503-355fb2ff3f4c admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:47.762780 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:32:47.767293 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab43a85-ce8d-4c16-9503-355fb2ff3f4c admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:47.771902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4de4e-c77d-4a3e-9445-8b8368e13623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:47.778064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4de4e-c77d-4a3e-9445-8b8368e13623 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:47.778064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4de4e-c77d-4a3e-9445-8b8368e13623 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:47.778064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4de4e-c77d-4a3e-9445-8b8368e13623 admin admin] PATH_INFO: `/v3/projects/8a79f3fe2a954504a155388b7780651d/users/f9ce84e784a246e4a7e79695a1f81365/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:47.788703 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4de4e-c77d-4a3e-9445-8b8368e13623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a79f3fe2a954504a155388b7780651d, user_id=f9ce84e784a246e4a7e79695a1f81365, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:47.791435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4de4e-c77d-4a3e-9445-8b8368e13623 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:47.889327 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1383/2804] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:32:47 2022] PUT /identity/v3/projects/8a79f3fe2a954504a155388b7780651d/users/f9ce84e784a246e4a7e79695a1f81365/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:32:47.894611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52f388-79c5-4558-bfed-5fcadf109597 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:47.894976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52f388-79c5-4558-bfed-5fcadf109597 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:47.895261 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52f388-79c5-4558-bfed-5fcadf109597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:47.914933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-df52f388-79c5-4558-bfed-5fcadf109597 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:32:47.916167 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-df52f388-79c5-4558-bfed-5fcadf109597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:47.976126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1422/2805] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 19:32:47.986595 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4de4e-c77d-4a3e-9445-8b8368e13623 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:47.988303 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-24a4de4e-c77d-4a3e-9445-8b8368e13623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:48.044742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4de4e-c77d-4a3e-9445-8b8368e13623 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:48.049066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0f77f-d643-42a4-b389-cbd038e3ee25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:48.049675 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f77f-d643-42a4-b389-cbd038e3ee25 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:48.049952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f77f-d643-42a4-b389-cbd038e3ee25 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:48.050216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f77f-d643-42a4-b389-cbd038e3ee25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:48.059075 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0f77f-d643-42a4-b389-cbd038e3ee25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:48.060490 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0f77f-d643-42a4-b389-cbd038e3ee25 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:48.069881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1384/2806] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 19:32:53.358446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1ee50-b177-48d5-af46-21fd94c984df None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:53.358838 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1ee50-b177-48d5-af46-21fd94c984df None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:53.358999 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1ee50-b177-48d5-af46-21fd94c984df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:53.392712 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:32:53.398911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-76d1ee50-b177-48d5-af46-21fd94c984df None None] MFA Rules not processed for user `f9ce84e784a246e4a7e79695a1f81365`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:32:53.400225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-76d1ee50-b177-48d5-af46-21fd94c984df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:53.482483 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1423/2807] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:32:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:32:53.483436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0f77f-d643-42a4-b389-cbd038e3ee25 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:53.493988 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0f77f-d643-42a4-b389-cbd038e3ee25 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:53.499258 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ecb5-e5ef-4b92-b69e-28ca4b70a5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:53.500528 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ecb5-e5ef-4b92-b69e-28ca4b70a5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:53.500820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ecb5-e5ef-4b92-b69e-28ca4b70a5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:53.501132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ecb5-e5ef-4b92-b69e-28ca4b70a5b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:53.501810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2ecb5-e5ef-4b92-b69e-28ca4b70a5b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:53.503905 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2ecb5-e5ef-4b92-b69e-28ca4b70a5b1 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:53.618245 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1385/2808] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:32:53 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:32:53.622229 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1ee50-b177-48d5-af46-21fd94c984df None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:53.634575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1ee50-b177-48d5-af46-21fd94c984df None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:53.707984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d22684-fc1c-405d-b226-7c5e4de0309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:53.708825 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d22684-fc1c-405d-b226-7c5e4de0309a admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:53.709275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d22684-fc1c-405d-b226-7c5e4de0309a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:53.709688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d22684-fc1c-405d-b226-7c5e4de0309a admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:53.710538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d22684-fc1c-405d-b226-7c5e4de0309a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:53.712708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d22684-fc1c-405d-b226-7c5e4de0309a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:53.824578 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1424/2809] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:32:53 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:32:53.829734 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ecb5-e5ef-4b92-b69e-28ca4b70a5b1 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:53.841403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ecb5-e5ef-4b92-b69e-28ca4b70a5b1 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:53.846367 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823d3fe-c93d-4a6f-a121-0af8d6333399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:53.847268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823d3fe-c93d-4a6f-a121-0af8d6333399 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:53.847583 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823d3fe-c93d-4a6f-a121-0af8d6333399 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:53.847860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823d3fe-c93d-4a6f-a121-0af8d6333399 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:53.848514 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823d3fe-c93d-4a6f-a121-0af8d6333399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:53.850156 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823d3fe-c93d-4a6f-a121-0af8d6333399 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:53.862065 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1386/2810] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:32:53.866541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d22684-fc1c-405d-b226-7c5e4de0309a admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:53.878649 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d22684-fc1c-405d-b226-7c5e4de0309a admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:53.884689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364afa72-4997-4b3c-9f39-e6c0c5b4b4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:53.885389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364afa72-4997-4b3c-9f39-e6c0c5b4b4db admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:53.885675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364afa72-4997-4b3c-9f39-e6c0c5b4b4db admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:53.885954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364afa72-4997-4b3c-9f39-e6c0c5b4b4db admin admin] PATH_INFO: `/v3/projects/3a99610fdc2b483784e420adb57ba5b9/users/247116c6453941b58cfbf49edda03b01/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:53.904337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364afa72-4997-4b3c-9f39-e6c0c5b4b4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a99610fdc2b483784e420adb57ba5b9, user_id=247116c6453941b58cfbf49edda03b01, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:53.907793 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364afa72-4997-4b3c-9f39-e6c0c5b4b4db admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:53.992392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1425/2811] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:32:53 2022] PUT /identity/v3/projects/3a99610fdc2b483784e420adb57ba5b9/users/247116c6453941b58cfbf49edda03b01/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:32:53.996451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823d3fe-c93d-4a6f-a121-0af8d6333399 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:54.007167 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823d3fe-c93d-4a6f-a121-0af8d6333399 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:54.065483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7aba5-7d21-4d52-86de-89f90f512ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:54.066095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7aba5-7d21-4d52-86de-89f90f512ddb admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:54.066377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7aba5-7d21-4d52-86de-89f90f512ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:54.066651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7aba5-7d21-4d52-86de-89f90f512ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:54.067290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7aba5-7d21-4d52-86de-89f90f512ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:54.068922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7aba5-7d21-4d52-86de-89f90f512ddb admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:54.079813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1387/2812] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:32:54.084143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364afa72-4997-4b3c-9f39-e6c0c5b4b4db admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:54.095052 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364afa72-4997-4b3c-9f39-e6c0c5b4b4db admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:54.099685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6346e-b1ed-4648-8d22-7bebb58f90b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:54.100332 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6346e-b1ed-4648-8d22-7bebb58f90b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:54.100611 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6346e-b1ed-4648-8d22-7bebb58f90b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:54.100884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6346e-b1ed-4648-8d22-7bebb58f90b6 admin admin] PATH_INFO: `/v3/projects/3a99610fdc2b483784e420adb57ba5b9/users/247116c6453941b58cfbf49edda03b01/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:54.102817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6346e-b1ed-4648-8d22-7bebb58f90b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a99610fdc2b483784e420adb57ba5b9, user_id=247116c6453941b58cfbf49edda03b01, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:54.105452 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6346e-b1ed-4648-8d22-7bebb58f90b6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:54.115867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1426/2813] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:32:54 2022] PUT /identity/v3/projects/3a99610fdc2b483784e420adb57ba5b9/users/247116c6453941b58cfbf49edda03b01/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:32:59.440877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca4f18-5cfe-47d4-8360-a55d4154c1db None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:59.441392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca4f18-5cfe-47d4-8360-a55d4154c1db None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:59.441392 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca4f18-5cfe-47d4-8360-a55d4154c1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:59.469684 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-00ca4f18-5cfe-47d4-8360-a55d4154c1db None None] MFA Rules not processed for user `247116c6453941b58cfbf49edda03b01`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:32:59.470906 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-00ca4f18-5cfe-47d4-8360-a55d4154c1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:59.506756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:32:59.549497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1388/2814] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:32:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:32:59.578057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6346e-b1ed-4648-8d22-7bebb58f90b6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:32:59.580062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d9d6346e-b1ed-4648-8d22-7bebb58f90b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:32:59.640732 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:32:59.660358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6346e-b1ed-4648-8d22-7bebb58f90b6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:32:59.667679 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14832ae-c1e1-4370-87c5-c326a37ba22e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:32:59.668313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14832ae-c1e1-4370-87c5-c326a37ba22e service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:32:59.668633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14832ae-c1e1-4370-87c5-c326a37ba22e service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:32:59.668850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14832ae-c1e1-4370-87c5-c326a37ba22e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:32:59.732904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14832ae-c1e1-4370-87c5-c326a37ba22e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:32:59.735333 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14832ae-c1e1-4370-87c5-c326a37ba22e service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:32:59.745851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1427/2815] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:32:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:09.595231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=13 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:09.824324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=31 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:19.307296 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4ca3d-8b36-4b37-81e2-7fceb629e660 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:19.307786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4ca3d-8b36-4b37-81e2-7fceb629e660 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:19.307889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4ca3d-8b36-4b37-81e2-7fceb629e660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:19.327368 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-1ab4ca3d-8b36-4b37-81e2-7fceb629e660 None None] MFA Rules not processed for user `f9ce84e784a246e4a7e79695a1f81365`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:33:19.329268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1ab4ca3d-8b36-4b37-81e2-7fceb629e660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:19.338336 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1389/2816] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:33:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:33:19.393094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14832ae-c1e1-4370-87c5-c326a37ba22e service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:19.394510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f14832ae-c1e1-4370-87c5-c326a37ba22e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:19.451059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14832ae-c1e1-4370-87c5-c326a37ba22e service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:19.454498 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a3143-e8fc-43b4-82db-be58e1418e42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:19.455099 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a3143-e8fc-43b4-82db-be58e1418e42 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:19.455375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a3143-e8fc-43b4-82db-be58e1418e42 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:19.455643 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a3143-e8fc-43b4-82db-be58e1418e42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:19.457341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2e0a3143-e8fc-43b4-82db-be58e1418e42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:19.471275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a3143-e8fc-43b4-82db-be58e1418e42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:19.472671 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a3143-e8fc-43b4-82db-be58e1418e42 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:19.483451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1428/2817] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:19.764123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4ca3d-8b36-4b37-81e2-7fceb629e660 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:19.819497 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4ca3d-8b36-4b37-81e2-7fceb629e660 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:19.823133 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959479d5-bfc5-4dbd-9741-95421fe1f675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:19.823755 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959479d5-bfc5-4dbd-9741-95421fe1f675 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:19.824022 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959479d5-bfc5-4dbd-9741-95421fe1f675 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:19.824300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959479d5-bfc5-4dbd-9741-95421fe1f675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:19.826080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-959479d5-bfc5-4dbd-9741-95421fe1f675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:19.835636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959479d5-bfc5-4dbd-9741-95421fe1f675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:19.837089 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959479d5-bfc5-4dbd-9741-95421fe1f675 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:19.845115 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1390/2818] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:20.556564 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a3143-e8fc-43b4-82db-be58e1418e42 service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:20.558085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-2e0a3143-e8fc-43b4-82db-be58e1418e42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:20.572848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959479d5-bfc5-4dbd-9741-95421fe1f675 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:20.581849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959479d5-bfc5-4dbd-9741-95421fe1f675 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:20.587612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e1617-0b4f-4563-970d-cd0159c863a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:20.588236 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e1617-0b4f-4563-970d-cd0159c863a4 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:20.588996 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e1617-0b4f-4563-970d-cd0159c863a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:20.589309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e1617-0b4f-4563-970d-cd0159c863a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:20.591696 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-901e1617-0b4f-4563-970d-cd0159c863a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:20.608479 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e1617-0b4f-4563-970d-cd0159c863a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:20.611061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e1617-0b4f-4563-970d-cd0159c863a4 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:20.622473 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1391/2819] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 19:33:20.637440 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a3143-e8fc-43b4-82db-be58e1418e42 service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:20.642514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ae675-31cb-48d6-ab82-4cce53a124bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:20.643178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ae675-31cb-48d6-ab82-4cce53a124bc service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:20.643598 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ae675-31cb-48d6-ab82-4cce53a124bc service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:20.643877 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ae675-31cb-48d6-ab82-4cce53a124bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:20.645678 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ae0ae675-31cb-48d6-ab82-4cce53a124bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:20.659311 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ae675-31cb-48d6-ab82-4cce53a124bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:20.660884 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ae675-31cb-48d6-ab82-4cce53a124bc service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:20.672193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1429/2820] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 19:33:24.886893 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e1617-0b4f-4563-970d-cd0159c863a4 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:24.898192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e1617-0b4f-4563-970d-cd0159c863a4 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:24.901593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245851f-b196-443d-82b7-e00f5a68de0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:24.902205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245851f-b196-443d-82b7-e00f5a68de0d service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:24.902492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245851f-b196-443d-82b7-e00f5a68de0d service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:24.902770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245851f-b196-443d-82b7-e00f5a68de0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:24.905181 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2245851f-b196-443d-82b7-e00f5a68de0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:24.915605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245851f-b196-443d-82b7-e00f5a68de0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:24.917039 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245851f-b196-443d-82b7-e00f5a68de0d service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:24.925383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1392/2821] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:25.800551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ae675-31cb-48d6-ab82-4cce53a124bc service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:25.802133 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ae0ae675-31cb-48d6-ab82-4cce53a124bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:25.819070 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ae675-31cb-48d6-ab82-4cce53a124bc service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:25.826003 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407560d8-69b8-403c-a634-ad5c45e4161d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:25.826616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407560d8-69b8-403c-a634-ad5c45e4161d service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:25.826892 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407560d8-69b8-403c-a634-ad5c45e4161d service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:25.827171 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407560d8-69b8-403c-a634-ad5c45e4161d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:25.829111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-407560d8-69b8-403c-a634-ad5c45e4161d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:25.841521 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407560d8-69b8-403c-a634-ad5c45e4161d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:25.843111 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407560d8-69b8-403c-a634-ad5c45e4161d service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:25.854766 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1430/2822] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:34.921443 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=41 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:35.849451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=64 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:36.012611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245851f-b196-443d-82b7-e00f5a68de0d service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:36.028503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245851f-b196-443d-82b7-e00f5a68de0d service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:36.046175 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc79ad-0368-4c59-8a65-e6361edc7ac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:36.046799 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc79ad-0368-4c59-8a65-e6361edc7ac7 service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:36.047079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc79ad-0368-4c59-8a65-e6361edc7ac7 service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:36.047350 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc79ad-0368-4c59-8a65-e6361edc7ac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:36.052708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-5ebc79ad-0368-4c59-8a65-e6361edc7ac7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:36.062681 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc79ad-0368-4c59-8a65-e6361edc7ac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:36.064249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc79ad-0368-4c59-8a65-e6361edc7ac7 service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:36.075527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1393/2823] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:38.594660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407560d8-69b8-403c-a634-ad5c45e4161d service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:38.596063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-407560d8-69b8-403c-a634-ad5c45e4161d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:38.608641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407560d8-69b8-403c-a634-ad5c45e4161d service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:38.612247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb43345-283d-47ff-a66d-34fb1d965b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:38.612848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb43345-283d-47ff-a66d-34fb1d965b5f service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:38.613159 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb43345-283d-47ff-a66d-34fb1d965b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:38.613437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb43345-283d-47ff-a66d-34fb1d965b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:38.623191 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb43345-283d-47ff-a66d-34fb1d965b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:38.624601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb43345-283d-47ff-a66d-34fb1d965b5f service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:38.634770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1431/2824] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:40.483483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc79ad-0368-4c59-8a65-e6361edc7ac7 service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:40.495822 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc79ad-0368-4c59-8a65-e6361edc7ac7 service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:40.501627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6095fa9-227d-469a-b5f3-9376753190ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:40.503642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6095fa9-227d-469a-b5f3-9376753190ce service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:40.503642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6095fa9-227d-469a-b5f3-9376753190ce service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:40.503642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6095fa9-227d-469a-b5f3-9376753190ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:40.504697 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c6095fa9-227d-469a-b5f3-9376753190ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:40.518003 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6095fa9-227d-469a-b5f3-9376753190ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:40.519598 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6095fa9-227d-469a-b5f3-9376753190ce service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:40.531340 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1394/2825] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:47.135105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb43345-283d-47ff-a66d-34fb1d965b5f service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:47.153591 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb43345-283d-47ff-a66d-34fb1d965b5f service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:47.157507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c78aa8-91af-4cf5-a377-0e6fa2fb5b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:47.158104 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c78aa8-91af-4cf5-a377-0e6fa2fb5b2c service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:47.158387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c78aa8-91af-4cf5-a377-0e6fa2fb5b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:47.158662 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c78aa8-91af-4cf5-a377-0e6fa2fb5b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:47.160393 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f3c78aa8-91af-4cf5-a377-0e6fa2fb5b2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:47.234823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c78aa8-91af-4cf5-a377-0e6fa2fb5b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:47.236284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c78aa8-91af-4cf5-a377-0e6fa2fb5b2c service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:47.246161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1432/2826] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:50.526361 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=12 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:50.858504 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6095fa9-227d-469a-b5f3-9376753190ce service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:50.860030 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c6095fa9-227d-469a-b5f3-9376753190ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:50.886032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6095fa9-227d-469a-b5f3-9376753190ce service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:50.890594 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:50.891229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:50.891517 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:50.891795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] PATH_INFO: `/v3/users/1c04b545aa674beab518178a8936b788` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:50.892913 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c04b545aa674beab518178a8936b788)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:50.894878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:51.073935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c04b545aa674beab518178a8936b788'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:51.074580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c04b545aa674beab518178a8936b788'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:51.075184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c04b545aa674beab518178a8936b788'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:51.088534 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c04b545aa674beab518178a8936b788'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:51.095639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f9fb801c-d3e8-4dca-ab5d-4bbcfc129291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c04b545aa674beab518178a8936b788'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:51.191387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1395/2827] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:33:50 2022] DELETE /identity/v3/users/1c04b545aa674beab518178a8936b788 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:33:51.451273 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c78aa8-91af-4cf5-a377-0e6fa2fb5b2c service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:51.462172 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c78aa8-91af-4cf5-a377-0e6fa2fb5b2c service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:51.522708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:51.523357 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:51.523648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:51.523925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] PATH_INFO: `/v3/projects/e786d60ca59f4d23be47a4a8305123f4` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:51.525091 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e786d60ca59f4d23be47a4a8305123f4)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:51.526994 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:51.626075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e786d60ca59f4d23be47a4a8305123f4', 'request_id': 'req-add820fa-46b3-48f0-a99e-afd84b7e5069'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:51.632178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e786d60ca59f4d23be47a4a8305123f4', 'request_id': 'req-add820fa-46b3-48f0-a99e-afd84b7e5069'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:51.721270 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e786d60ca59f4d23be47a4a8305123f4', 'request_id': 'req-add820fa-46b3-48f0-a99e-afd84b7e5069'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:51.725886 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:51.733750 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] The token cache is being invalidate because project e786d60ca59f4d23be47a4a8305123f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:33:51.734039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:51.734680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-add820fa-46b3-48f0-a99e-afd84b7e5069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:51.736201 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1433/2828] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:33:51 2022] DELETE /identity/v3/projects/e786d60ca59f4d23be47a4a8305123f4 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:33:51.742126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac0f8d-a45d-4589-b723-aa8532e05f6e None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:51.742607 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac0f8d-a45d-4589-b723-aa8532e05f6e None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:51.742815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac0f8d-a45d-4589-b723-aa8532e05f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:51.763565 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-6cac0f8d-a45d-4589-b723-aa8532e05f6e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:33:51.764860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6cac0f8d-a45d-4589-b723-aa8532e05f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:51.826519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1396/2829] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:33:51.836168 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c50e3b-ffee-438d-805b-8b8574b36537 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:51.836451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c50e3b-ffee-438d-805b-8b8574b36537 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:51.841151 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c50e3b-ffee-438d-805b-8b8574b36537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:51.856480 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-22c50e3b-ffee-438d-805b-8b8574b36537 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:33:51.857713 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-22c50e3b-ffee-438d-805b-8b8574b36537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:51.865772 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1434/2830] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:33:51.870856 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac0f8d-a45d-4589-b723-aa8532e05f6e None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:51.880774 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac0f8d-a45d-4589-b723-aa8532e05f6e None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:51.886452 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b96c33-31be-43f3-a27d-53e41b926591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:51.887084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b96c33-31be-43f3-a27d-53e41b926591 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:51.887366 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b96c33-31be-43f3-a27d-53e41b926591 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:51.887641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b96c33-31be-43f3-a27d-53e41b926591 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:51.888285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b96c33-31be-43f3-a27d-53e41b926591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 19:33:51.888592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b96c33-31be-43f3-a27d-53e41b926591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:51.890224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b96c33-31be-43f3-a27d-53e41b926591 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:51.903668 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1397/2831] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 19:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 19:33:51.912946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c50e3b-ffee-438d-805b-8b8574b36537 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:51.922309 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c50e3b-ffee-438d-805b-8b8574b36537 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:51.926946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317c91e-5131-44ba-8fc8-0e7f42503bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:51.927593 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317c91e-5131-44ba-8fc8-0e7f42503bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:51.927872 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317c91e-5131-44ba-8fc8-0e7f42503bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:51.928138 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317c91e-5131-44ba-8fc8-0e7f42503bc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:51.928775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317c91e-5131-44ba-8fc8-0e7f42503bc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:51.931367 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317c91e-5131-44ba-8fc8-0e7f42503bc5 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:52.015981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1435/2832] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:33:51 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:33:52.020337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b96c33-31be-43f3-a27d-53e41b926591 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:52.031563 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b96c33-31be-43f3-a27d-53e41b926591 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:52.085238 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:52.095719 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d58e41-a546-4f32-bd9f-9bd55ece5adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:52.096330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d58e41-a546-4f32-bd9f-9bd55ece5adb admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:52.096612 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d58e41-a546-4f32-bd9f-9bd55ece5adb admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:52.096889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d58e41-a546-4f32-bd9f-9bd55ece5adb admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:52.097570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d58e41-a546-4f32-bd9f-9bd55ece5adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:52.099492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d58e41-a546-4f32-bd9f-9bd55ece5adb admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:52.189380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1398/2833] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:33:52 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:33:52.193528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317c91e-5131-44ba-8fc8-0e7f42503bc5 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:52.204698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317c91e-5131-44ba-8fc8-0e7f42503bc5 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:52.209297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a271cc0-1c85-4560-bda5-cb250b834451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:52.209898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a271cc0-1c85-4560-bda5-cb250b834451 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:52.210183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a271cc0-1c85-4560-bda5-cb250b834451 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:52.210460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a271cc0-1c85-4560-bda5-cb250b834451 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:52.211940 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a271cc0-1c85-4560-bda5-cb250b834451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:52.214308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a271cc0-1c85-4560-bda5-cb250b834451 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:52.226140 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1436/2834] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:33:52.230430 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d58e41-a546-4f32-bd9f-9bd55ece5adb admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:52.240650 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d58e41-a546-4f32-bd9f-9bd55ece5adb admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:52.245652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09ccb4-16be-421c-9ccc-2bd08c827386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:52.246249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09ccb4-16be-421c-9ccc-2bd08c827386 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:52.246531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09ccb4-16be-421c-9ccc-2bd08c827386 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:52.246832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09ccb4-16be-421c-9ccc-2bd08c827386 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:52.247912 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09ccb4-16be-421c-9ccc-2bd08c827386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:52.250069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09ccb4-16be-421c-9ccc-2bd08c827386 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:52.261603 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1399/2835] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:33:52.265723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a271cc0-1c85-4560-bda5-cb250b834451 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:52.276484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a271cc0-1c85-4560-bda5-cb250b834451 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:52.281389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875895a9-961f-41a6-8fcc-72b953f8e685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:52.282039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875895a9-961f-41a6-8fcc-72b953f8e685 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:52.282322 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875895a9-961f-41a6-8fcc-72b953f8e685 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:52.282600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875895a9-961f-41a6-8fcc-72b953f8e685 admin admin] PATH_INFO: `/v3/projects/8fc0d59a922d4e5b96d7fab853ffbb45/users/837c361e114a4973803f7e1362bfff5d/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:52.297455 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875895a9-961f-41a6-8fcc-72b953f8e685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc0d59a922d4e5b96d7fab853ffbb45, user_id=837c361e114a4973803f7e1362bfff5d, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:52.300126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875895a9-961f-41a6-8fcc-72b953f8e685 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:52.396997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1437/2836] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:33:52 2022] PUT /identity/v3/projects/8fc0d59a922d4e5b96d7fab853ffbb45/users/837c361e114a4973803f7e1362bfff5d/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:33:52.408379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7cf6c-d755-4194-be5c-c27443e37e11 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:52.408666 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7cf6c-d755-4194-be5c-c27443e37e11 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:52.408947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7cf6c-d755-4194-be5c-c27443e37e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:52.438407 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-fdd7cf6c-d755-4194-be5c-c27443e37e11 None None] MFA Rules not processed for user `837c361e114a4973803f7e1362bfff5d`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:33:52.439756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-fdd7cf6c-d755-4194-be5c-c27443e37e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:52.503608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1400/2837] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:33:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 19:33:52.508843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875895a9-961f-41a6-8fcc-72b953f8e685 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:52.572359 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875895a9-961f-41a6-8fcc-72b953f8e685 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:52.576916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d16a9a-4e3d-4fab-b9c4-af52080cfd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:52.577538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d16a9a-4e3d-4fab-b9c4-af52080cfd03 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:52.577815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d16a9a-4e3d-4fab-b9c4-af52080cfd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:52.578082 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d16a9a-4e3d-4fab-b9c4-af52080cfd03 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:52.578699 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d16a9a-4e3d-4fab-b9c4-af52080cfd03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:52.580614 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d16a9a-4e3d-4fab-b9c4-af52080cfd03 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:52.693956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1438/2838] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:33:52 2022] POST /identity/v3/projects => generated 363 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:33:52.698887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7cf6c-d755-4194-be5c-c27443e37e11 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:52.709656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7cf6c-d755-4194-be5c-c27443e37e11 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:52.765132 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f436e-b62c-4276-a3f2-c41d5b622450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:52.765784 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f436e-b62c-4276-a3f2-c41d5b622450 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:52.765997 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f436e-b62c-4276-a3f2-c41d5b622450 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:52.766267 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f436e-b62c-4276-a3f2-c41d5b622450 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:52.766883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f436e-b62c-4276-a3f2-c41d5b622450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:52.768783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f436e-b62c-4276-a3f2-c41d5b622450 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:52.874979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1401/2839] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:33:52 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:33:52.879059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d16a9a-4e3d-4fab-b9c4-af52080cfd03 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:52.890113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d16a9a-4e3d-4fab-b9c4-af52080cfd03 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:52.894945 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f32c64-98c3-4746-a62e-b4e8a9f3c5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:52.895535 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f32c64-98c3-4746-a62e-b4e8a9f3c5cd admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:52.895810 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f32c64-98c3-4746-a62e-b4e8a9f3c5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:52.896080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f32c64-98c3-4746-a62e-b4e8a9f3c5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:52.896826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f32c64-98c3-4746-a62e-b4e8a9f3c5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:52.899247 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f32c64-98c3-4746-a62e-b4e8a9f3c5cd admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:52.910142 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1439/2840] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:33:52.915000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f436e-b62c-4276-a3f2-c41d5b622450 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:52.924790 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f436e-b62c-4276-a3f2-c41d5b622450 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:52.929331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62158339-b2d0-499c-9816-ea0edc22dbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:52.930077 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62158339-b2d0-499c-9816-ea0edc22dbec admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:52.930352 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62158339-b2d0-499c-9816-ea0edc22dbec admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:52.930623 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62158339-b2d0-499c-9816-ea0edc22dbec admin admin] PATH_INFO: `/v3/projects/d9524363b05446d5943cd8d9d0d9e58e/users/60942a9c022d44048d7ffa03126ca0e4/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:52.945846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62158339-b2d0-499c-9816-ea0edc22dbec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9524363b05446d5943cd8d9d0d9e58e, user_id=60942a9c022d44048d7ffa03126ca0e4, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:52.948440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62158339-b2d0-499c-9816-ea0edc22dbec admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:53.006146 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1402/2841] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:33:52 2022] PUT /identity/v3/projects/d9524363b05446d5943cd8d9d0d9e58e/users/60942a9c022d44048d7ffa03126ca0e4/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:33:53.010727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f32c64-98c3-4746-a62e-b4e8a9f3c5cd admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:53.021072 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f32c64-98c3-4746-a62e-b4e8a9f3c5cd admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:53.076148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96cd19-830b-4b67-9b4a-9ef7c8059ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:53.076742 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96cd19-830b-4b67-9b4a-9ef7c8059ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:53.077020 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96cd19-830b-4b67-9b4a-9ef7c8059ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:53.077323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96cd19-830b-4b67-9b4a-9ef7c8059ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:53.077950 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96cd19-830b-4b67-9b4a-9ef7c8059ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:53.079514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96cd19-830b-4b67-9b4a-9ef7c8059ef6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:53.084382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:53.089954 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1440/2842] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:33:53.094130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62158339-b2d0-499c-9816-ea0edc22dbec admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:53.104483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62158339-b2d0-499c-9816-ea0edc22dbec admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:53.109207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29201549-cee3-4f34-a69b-4f65f09d0fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:53.109856 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29201549-cee3-4f34-a69b-4f65f09d0fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:53.110140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29201549-cee3-4f34-a69b-4f65f09d0fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:53.110420 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29201549-cee3-4f34-a69b-4f65f09d0fb7 admin admin] PATH_INFO: `/v3/projects/d9524363b05446d5943cd8d9d0d9e58e/users/60942a9c022d44048d7ffa03126ca0e4/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:53.112446 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29201549-cee3-4f34-a69b-4f65f09d0fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9524363b05446d5943cd8d9d0d9e58e, user_id=60942a9c022d44048d7ffa03126ca0e4, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:53.115116 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29201549-cee3-4f34-a69b-4f65f09d0fb7 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:53.125624 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1403/2843] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:33:53 2022] PUT /identity/v3/projects/d9524363b05446d5943cd8d9d0d9e58e/users/60942a9c022d44048d7ffa03126ca0e4/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:33:53.136511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b02e5-2359-4884-89d9-cf6d2f233853 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:53.136791 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b02e5-2359-4884-89d9-cf6d2f233853 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:53.137548 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b02e5-2359-4884-89d9-cf6d2f233853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:53.165415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-431b02e5-2359-4884-89d9-cf6d2f233853 None None] MFA Rules not processed for user `60942a9c022d44048d7ffa03126ca0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:33:53.166737 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-431b02e5-2359-4884-89d9-cf6d2f233853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:53.235761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1441/2844] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:33:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:33:53.245748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29201549-cee3-4f34-a69b-4f65f09d0fb7 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:53.247442 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-29201549-cee3-4f34-a69b-4f65f09d0fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:53.261459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:53.305415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29201549-cee3-4f34-a69b-4f65f09d0fb7 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:53.309173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965f3a3-7613-4f9f-b4f4-ac16391610c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:53.309777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965f3a3-7613-4f9f-b4f4-ac16391610c0 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:53.310060 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965f3a3-7613-4f9f-b4f4-ac16391610c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:53.310337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965f3a3-7613-4f9f-b4f4-ac16391610c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:53.319281 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965f3a3-7613-4f9f-b4f4-ac16391610c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:53.321001 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965f3a3-7613-4f9f-b4f4-ac16391610c0 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:53.331287 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1404/2845] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:53.537695 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b02e5-2359-4884-89d9-cf6d2f233853 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:53.548299 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b02e5-2359-4884-89d9-cf6d2f233853 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:53.551835 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390ba79-2cba-4c2d-aa0a-ff1eff67f079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:53.553651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390ba79-2cba-4c2d-aa0a-ff1eff67f079 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:53.553651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390ba79-2cba-4c2d-aa0a-ff1eff67f079 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:53.553651 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390ba79-2cba-4c2d-aa0a-ff1eff67f079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:53.610736 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390ba79-2cba-4c2d-aa0a-ff1eff67f079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:53.612169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390ba79-2cba-4c2d-aa0a-ff1eff67f079 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:53.621363 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1442/2846] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:57.114387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965f3a3-7613-4f9f-b4f4-ac16391610c0 service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:57.181588 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965f3a3-7613-4f9f-b4f4-ac16391610c0 service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:57.186686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:57.187311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:57.187639 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:57.187881 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] PATH_INFO: `/v3/users/837c361e114a4973803f7e1362bfff5d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:57.189104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=837c361e114a4973803f7e1362bfff5d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:57.191021 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:57.371306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837c361e114a4973803f7e1362bfff5d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:57.371964 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837c361e114a4973803f7e1362bfff5d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:57.372569 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837c361e114a4973803f7e1362bfff5d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:57.385061 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837c361e114a4973803f7e1362bfff5d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:57.391934 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837c361e114a4973803f7e1362bfff5d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:57.494458 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1405/2847] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:33:57 2022] DELETE /identity/v3/users/837c361e114a4973803f7e1362bfff5d => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:33:57.498925 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba1380-552c-4af0-bf2a-db5f10eac253 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:57.501590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba1380-552c-4af0-bf2a-db5f10eac253 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:57.501590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba1380-552c-4af0-bf2a-db5f10eac253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:57.519620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-e3ba1380-552c-4af0-bf2a-db5f10eac253 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:33:57.527339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e3ba1380-552c-4af0-bf2a-db5f10eac253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:57.580579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1443/2848] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:33:57.590945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:57.592360 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:57.630250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:57.661714 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235885f3-14fb-41de-8e17-13afd503a834 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:57.665376 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50169f2-8fa4-4955-a247-7236d431c635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:57.668333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50169f2-8fa4-4955-a247-7236d431c635 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:57.668333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50169f2-8fa4-4955-a247-7236d431c635 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:57.668333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50169f2-8fa4-4955-a247-7236d431c635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:57.675326 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50169f2-8fa4-4955-a247-7236d431c635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:57.676860 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50169f2-8fa4-4955-a247-7236d431c635 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:57.686527 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1406/2849] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:33:58.101406 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba1380-552c-4af0-bf2a-db5f10eac253 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:58.102998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-e3ba1380-552c-4af0-bf2a-db5f10eac253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:58.114976 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba1380-552c-4af0-bf2a-db5f10eac253 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:58.120193 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:58.120790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:58.121095 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:58.121377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] PATH_INFO: `/v3/projects/8fc0d59a922d4e5b96d7fab853ffbb45` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:58.122698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fc0d59a922d4e5b96d7fab853ffbb45)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:58.124601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:58.147705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=100 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:58.256505 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc0d59a922d4e5b96d7fab853ffbb45', 'request_id': 'req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:58.263280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc0d59a922d4e5b96d7fab853ffbb45', 'request_id': 'req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:58.346006 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc0d59a922d4e5b96d7fab853ffbb45', 'request_id': 'req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:58.359234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] The token cache is being invalidate because project 8fc0d59a922d4e5b96d7fab853ffbb45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:33:58.361551 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:58.361895 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:58.364076 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1444/2850] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:33:58 2022] DELETE /identity/v3/projects/8fc0d59a922d4e5b96d7fab853ffbb45 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:33:58.368138 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50169f2-8fa4-4955-a247-7236d431c635 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:58.382342 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50169f2-8fa4-4955-a247-7236d431c635 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:58.446295 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:58.446922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:58.447205 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:58.447484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] PATH_INFO: `/v3/users/60942a9c022d44048d7ffa03126ca0e4` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:58.448611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60942a9c022d44048d7ffa03126ca0e4)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:58.450560 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:58.638328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60942a9c022d44048d7ffa03126ca0e4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:58.639095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60942a9c022d44048d7ffa03126ca0e4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:58.639762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60942a9c022d44048d7ffa03126ca0e4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:58.652249 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60942a9c022d44048d7ffa03126ca0e4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:58.659602 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-fac817ba-38e5-4730-a6da-9d19672f70b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60942a9c022d44048d7ffa03126ca0e4'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:58.738139 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1407/2851] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:33:58 2022] DELETE /identity/v3/users/60942a9c022d44048d7ffa03126ca0e4 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 19:33:59.147654 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:59.151807 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:59.167364 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff36c8e-5adf-4621-8d25-8798b7bd1e31 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:59.226119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:59.226788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:59.227071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:59.227345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] PATH_INFO: `/v3/projects/d9524363b05446d5943cd8d9d0d9e58e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:59.228459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9524363b05446d5943cd8d9d0d9e58e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:59.230402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:59.317543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9524363b05446d5943cd8d9d0d9e58e', 'request_id': 'req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:59.324341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9524363b05446d5943cd8d9d0d9e58e', 'request_id': 'req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:59.391407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9524363b05446d5943cd8d9d0d9e58e', 'request_id': 'req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:59.402479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] The token cache is being invalidate because project d9524363b05446d5943cd8d9d0d9e58e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:33:59.409947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:59.409947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-62c1827a-6aa2-4ef0-8fe3-ca43921c9575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:33:59.409947 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1445/2852] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:33:59 2022] DELETE /identity/v3/projects/d9524363b05446d5943cd8d9d0d9e58e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:33:59.410351 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1e99e-2b1b-4002-9a51-9a2f44ff50a3 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:59.410897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1e99e-2b1b-4002-9a51-9a2f44ff50a3 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:59.410897 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1e99e-2b1b-4002-9a51-9a2f44ff50a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:59.431000 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-0df1e99e-2b1b-4002-9a51-9a2f44ff50a3 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:33:59.432141 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0df1e99e-2b1b-4002-9a51-9a2f44ff50a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:59.472289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:59.490942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1408/2853] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:33:59.500211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94230865-d1cc-4bd3-a0d7-44c3e654f4e9 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:59.500496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94230865-d1cc-4bd3-a0d7-44c3e654f4e9 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:59.500769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94230865-d1cc-4bd3-a0d7-44c3e654f4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:59.520868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-94230865-d1cc-4bd3-a0d7-44c3e654f4e9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:33:59.522117 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-94230865-d1cc-4bd3-a0d7-44c3e654f4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:33:59.531341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1446/2854] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:33:59.535459 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1e99e-2b1b-4002-9a51-9a2f44ff50a3 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:59.546101 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1e99e-2b1b-4002-9a51-9a2f44ff50a3 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:59.550742 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869c14a-c21a-420d-b696-b378a362fb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:59.551339 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c14a-c21a-420d-b696-b378a362fb39 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:59.551622 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c14a-c21a-420d-b696-b378a362fb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:59.551898 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869c14a-c21a-420d-b696-b378a362fb39 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:59.552540 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869c14a-c21a-420d-b696-b378a362fb39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 19:33:59.552849 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869c14a-c21a-420d-b696-b378a362fb39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:59.554502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869c14a-c21a-420d-b696-b378a362fb39 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:59.567432 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1409/2855] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 19:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 19:33:59.571823 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94230865-d1cc-4bd3-a0d7-44c3e654f4e9 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:59.581420 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94230865-d1cc-4bd3-a0d7-44c3e654f4e9 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:59.586613 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1405e-e8bc-49f4-8b02-f04474203fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:59.587279 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1405e-e8bc-49f4-8b02-f04474203fab admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:59.587557 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1405e-e8bc-49f4-8b02-f04474203fab admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:59.587827 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1405e-e8bc-49f4-8b02-f04474203fab admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:59.588463 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1405e-e8bc-49f4-8b02-f04474203fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:59.590428 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1405e-e8bc-49f4-8b02-f04474203fab admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:59.665641 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1447/2856] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:33:59 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:33:59.669902 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869c14a-c21a-420d-b696-b378a362fb39 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:59.679965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869c14a-c21a-420d-b696-b378a362fb39 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:59.741118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dced4bc-fabd-4b1a-9f9d-fd3b7d899537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:59.741651 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dced4bc-fabd-4b1a-9f9d-fd3b7d899537 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:59.741940 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dced4bc-fabd-4b1a-9f9d-fd3b7d899537 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:59.742379 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dced4bc-fabd-4b1a-9f9d-fd3b7d899537 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:59.743058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dced4bc-fabd-4b1a-9f9d-fd3b7d899537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:59.744984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dced4bc-fabd-4b1a-9f9d-fd3b7d899537 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:59.853278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1410/2857] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:33:59 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:33:59.857415 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1405e-e8bc-49f4-8b02-f04474203fab admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:59.868430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1405e-e8bc-49f4-8b02-f04474203fab admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:59.873148 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b1a60-2625-44f2-bef1-8748d6def54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:59.873747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b1a60-2625-44f2-bef1-8748d6def54e admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:59.874029 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b1a60-2625-44f2-bef1-8748d6def54e admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:59.874304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b1a60-2625-44f2-bef1-8748d6def54e admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:59.874946 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b1a60-2625-44f2-bef1-8748d6def54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:59.876622 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b1a60-2625-44f2-bef1-8748d6def54e admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:59.879013 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:33:59.888407 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1448/2858] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:33:59.892372 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dced4bc-fabd-4b1a-9f9d-fd3b7d899537 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:59.902826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dced4bc-fabd-4b1a-9f9d-fd3b7d899537 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:59.908253 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844bdcb-8368-419a-b74b-5bfedb854be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:59.913250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844bdcb-8368-419a-b74b-5bfedb854be4 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:59.913250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844bdcb-8368-419a-b74b-5bfedb854be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:59.913250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844bdcb-8368-419a-b74b-5bfedb854be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:59.913250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844bdcb-8368-419a-b74b-5bfedb854be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:59.913250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844bdcb-8368-419a-b74b-5bfedb854be4 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:33:59.923288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1411/2859] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:33:59.927418 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b1a60-2625-44f2-bef1-8748d6def54e admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:33:59.938478 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b1a60-2625-44f2-bef1-8748d6def54e admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:33:59.943049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71d8d5-2d7b-40d6-a0bd-36a17a019da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:33:59.943705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71d8d5-2d7b-40d6-a0bd-36a17a019da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:33:59.943986 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71d8d5-2d7b-40d6-a0bd-36a17a019da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:33:59.944260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71d8d5-2d7b-40d6-a0bd-36a17a019da8 admin admin] PATH_INFO: `/v3/projects/9809daeb0f594e26b33716b7c47b12c9/users/4965ba844059402cbd44aa25e7085aef/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:33:59.958761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71d8d5-2d7b-40d6-a0bd-36a17a019da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9809daeb0f594e26b33716b7c47b12c9, user_id=4965ba844059402cbd44aa25e7085aef, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:33:59.961588 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71d8d5-2d7b-40d6-a0bd-36a17a019da8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:00.020361 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1449/2860] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:33:59 2022] PUT /identity/v3/projects/9809daeb0f594e26b33716b7c47b12c9/users/4965ba844059402cbd44aa25e7085aef/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:00.032106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29f68f-7924-4c3d-a197-7bcc0fda7cbb None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:00.032398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29f68f-7924-4c3d-a197-7bcc0fda7cbb None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:00.032679 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29f68f-7924-4c3d-a197-7bcc0fda7cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:00.060593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-3b29f68f-7924-4c3d-a197-7bcc0fda7cbb None None] MFA Rules not processed for user `4965ba844059402cbd44aa25e7085aef`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:00.061882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-3b29f68f-7924-4c3d-a197-7bcc0fda7cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:00.125129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1412/2861] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:00.129574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71d8d5-2d7b-40d6-a0bd-36a17a019da8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:00.190920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71d8d5-2d7b-40d6-a0bd-36a17a019da8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:00.196493 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f679ff-d19d-4e12-b1f7-ca9a44e95221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:00.197113 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f679ff-d19d-4e12-b1f7-ca9a44e95221 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:00.197427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f679ff-d19d-4e12-b1f7-ca9a44e95221 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:00.197697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f679ff-d19d-4e12-b1f7-ca9a44e95221 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:00.198377 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f679ff-d19d-4e12-b1f7-ca9a44e95221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:00.200264 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f679ff-d19d-4e12-b1f7-ca9a44e95221 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:00.284565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1450/2862] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:34:00 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:34:00.288567 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29f68f-7924-4c3d-a197-7bcc0fda7cbb None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:00.299622 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29f68f-7924-4c3d-a197-7bcc0fda7cbb None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:00.338882 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:00.357566 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce2121-a39b-4da7-8381-12bd3cecf349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:00.358174 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce2121-a39b-4da7-8381-12bd3cecf349 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:00.358457 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce2121-a39b-4da7-8381-12bd3cecf349 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:00.358732 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce2121-a39b-4da7-8381-12bd3cecf349 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:00.359354 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce2121-a39b-4da7-8381-12bd3cecf349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:00.361286 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce2121-a39b-4da7-8381-12bd3cecf349 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:00.437200 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1413/2863] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:34:00 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:34:00.441347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f679ff-d19d-4e12-b1f7-ca9a44e95221 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:00.452553 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f679ff-d19d-4e12-b1f7-ca9a44e95221 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:00.457280 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f6233-bbbe-4a2a-bc0d-d97d4f4f74a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:00.457879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f6233-bbbe-4a2a-bc0d-d97d4f4f74a1 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:00.458162 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f6233-bbbe-4a2a-bc0d-d97d4f4f74a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:00.458437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f6233-bbbe-4a2a-bc0d-d97d4f4f74a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:00.459074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f6233-bbbe-4a2a-bc0d-d97d4f4f74a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:00.460645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f6233-bbbe-4a2a-bc0d-d97d4f4f74a1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:00.472532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1451/2864] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 19:34:00.476671 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce2121-a39b-4da7-8381-12bd3cecf349 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:00.486768 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce2121-a39b-4da7-8381-12bd3cecf349 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:00.491422 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c373b60-9436-496e-83ae-186f83c8b2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:00.494288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c373b60-9436-496e-83ae-186f83c8b2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:00.494288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c373b60-9436-496e-83ae-186f83c8b2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:00.494288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c373b60-9436-496e-83ae-186f83c8b2d5 admin admin] PATH_INFO: `/v3/projects/5f9fdf5b71544afc99b57bd137b2e419/users/18f1ad1860ae43bab357d4a532fe6e07/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:00.508645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c373b60-9436-496e-83ae-186f83c8b2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9fdf5b71544afc99b57bd137b2e419, user_id=18f1ad1860ae43bab357d4a532fe6e07, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:00.511299 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c373b60-9436-496e-83ae-186f83c8b2d5 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:00.574606 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1414/2865] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:00 2022] PUT /identity/v3/projects/5f9fdf5b71544afc99b57bd137b2e419/users/18f1ad1860ae43bab357d4a532fe6e07/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 19:34:00.578526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f6233-bbbe-4a2a-bc0d-d97d4f4f74a1 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:00.589790 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f6233-bbbe-4a2a-bc0d-d97d4f4f74a1 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:00.647192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbda71-97a6-440f-94c9-bb0028150c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:00.647801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbda71-97a6-440f-94c9-bb0028150c1a admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:00.648080 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbda71-97a6-440f-94c9-bb0028150c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:00.649119 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbda71-97a6-440f-94c9-bb0028150c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:00.649800 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbda71-97a6-440f-94c9-bb0028150c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:00.651379 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbda71-97a6-440f-94c9-bb0028150c1a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:00.661486 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1452/2866] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:00.665785 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c373b60-9436-496e-83ae-186f83c8b2d5 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:00.676170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c373b60-9436-496e-83ae-186f83c8b2d5 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:00.681231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9112c4-b421-4a88-bbb2-f27788950a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:00.681877 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9112c4-b421-4a88-bbb2-f27788950a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:00.682159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9112c4-b421-4a88-bbb2-f27788950a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:00.682431 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9112c4-b421-4a88-bbb2-f27788950a39 admin admin] PATH_INFO: `/v3/projects/5f9fdf5b71544afc99b57bd137b2e419/users/18f1ad1860ae43bab357d4a532fe6e07/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:00.684357 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9112c4-b421-4a88-bbb2-f27788950a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9fdf5b71544afc99b57bd137b2e419, user_id=18f1ad1860ae43bab357d4a532fe6e07, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:00.687028 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9112c4-b421-4a88-bbb2-f27788950a39 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:00.698014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1415/2867] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:00 2022] PUT /identity/v3/projects/5f9fdf5b71544afc99b57bd137b2e419/users/18f1ad1860ae43bab357d4a532fe6e07/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:00.708057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee6508-0453-46f5-9a79-f20ce276deca None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:00.708347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee6508-0453-46f5-9a79-f20ce276deca None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:00.708627 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee6508-0453-46f5-9a79-f20ce276deca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:00.736770 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1cee6508-0453-46f5-9a79-f20ce276deca None None] MFA Rules not processed for user `18f1ad1860ae43bab357d4a532fe6e07`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:00.738053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1cee6508-0453-46f5-9a79-f20ce276deca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:00.771242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:00.807323 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1453/2868] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:00.818483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9112c4-b421-4a88-bbb2-f27788950a39 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:00.820182 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-0f9112c4-b421-4a88-bbb2-f27788950a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:00.879189 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9112c4-b421-4a88-bbb2-f27788950a39 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:00.882948 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfd2a5-9cdd-4262-a48b-2b09a04e6245 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:00.883536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfd2a5-9cdd-4262-a48b-2b09a04e6245 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:00.883815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfd2a5-9cdd-4262-a48b-2b09a04e6245 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:00.884086 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfd2a5-9cdd-4262-a48b-2b09a04e6245 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:00.893382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfd2a5-9cdd-4262-a48b-2b09a04e6245 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:00.894770 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfd2a5-9cdd-4262-a48b-2b09a04e6245 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:00.905450 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1416/2869] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:00.979792 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee6508-0453-46f5-9a79-f20ce276deca None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:01.042904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee6508-0453-46f5-9a79-f20ce276deca None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:01.047511 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:01.048145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:01.048423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:01.048697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] PATH_INFO: `/v3/users/4965ba844059402cbd44aa25e7085aef` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:01.049859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4965ba844059402cbd44aa25e7085aef)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:01.051777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:01.212029 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4965ba844059402cbd44aa25e7085aef'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:01.212653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4965ba844059402cbd44aa25e7085aef'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:01.212822 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4965ba844059402cbd44aa25e7085aef'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:01.219839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4965ba844059402cbd44aa25e7085aef'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:01.232886 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4965ba844059402cbd44aa25e7085aef'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:01.310887 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1454/2870] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:34:00 2022] DELETE /identity/v3/users/4965ba844059402cbd44aa25e7085aef => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:01.315378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427a799-2d22-425e-b65a-7448a7b8200f None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:01.315687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427a799-2d22-425e-b65a-7448a7b8200f None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:01.315970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427a799-2d22-425e-b65a-7448a7b8200f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:01.336952 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-d427a799-2d22-425e-b65a-7448a7b8200f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:01.338403 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d427a799-2d22-425e-b65a-7448a7b8200f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:01.401211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1417/2871] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:01.411179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:01.412576 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:01.467644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:01.471410 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcdcd4-e25b-48ec-acd8-a35fa86e976b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:01.475049 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850113b8-9f3c-40a2-ad9f-4c09f9370056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:01.475655 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850113b8-9f3c-40a2-ad9f-4c09f9370056 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:01.475938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850113b8-9f3c-40a2-ad9f-4c09f9370056 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:01.476213 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850113b8-9f3c-40a2-ad9f-4c09f9370056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:01.485756 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850113b8-9f3c-40a2-ad9f-4c09f9370056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:01.487177 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850113b8-9f3c-40a2-ad9f-4c09f9370056 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:01.496706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1455/2872] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:01.878899 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427a799-2d22-425e-b65a-7448a7b8200f None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:01.880323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d427a799-2d22-425e-b65a-7448a7b8200f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:01.893084 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427a799-2d22-425e-b65a-7448a7b8200f None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:01.897787 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:01.899042 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:01.899359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:01.899629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] PATH_INFO: `/v3/projects/9809daeb0f594e26b33716b7c47b12c9` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:01.901137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9809daeb0f594e26b33716b7c47b12c9)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:01.902923 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:01.979468 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:01.992704 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9809daeb0f594e26b33716b7c47b12c9', 'request_id': 'req-d71becc0-7810-4203-a6a1-bc3b8a65135d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:01.999002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9809daeb0f594e26b33716b7c47b12c9', 'request_id': 'req-d71becc0-7810-4203-a6a1-bc3b8a65135d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:02.004613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9809daeb0f594e26b33716b7c47b12c9', 'request_id': 'req-d71becc0-7810-4203-a6a1-bc3b8a65135d'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:02.091649 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] The token cache is being invalidate because project 9809daeb0f594e26b33716b7c47b12c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:34:02.091944 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:02.092580 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:02.094162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1418/2873] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:34:01 2022] DELETE /identity/v3/projects/9809daeb0f594e26b33716b7c47b12c9 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:02.098287 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850113b8-9f3c-40a2-ad9f-4c09f9370056 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:02.108834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850113b8-9f3c-40a2-ad9f-4c09f9370056 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:02.168169 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:02.168804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:02.169116 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:02.169397 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] PATH_INFO: `/v3/users/18f1ad1860ae43bab357d4a532fe6e07` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:02.170519 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f1ad1860ae43bab357d4a532fe6e07)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:02.172426 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:02.357389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f1ad1860ae43bab357d4a532fe6e07'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:02.358021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f1ad1860ae43bab357d4a532fe6e07'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:02.358625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f1ad1860ae43bab357d4a532fe6e07'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:02.365879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f1ad1860ae43bab357d4a532fe6e07'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:02.378949 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-64f11dde-3e70-416b-932b-3a5f8ce9675f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f1ad1860ae43bab357d4a532fe6e07'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:02.463914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1456/2874] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:34:02 2022] DELETE /identity/v3/users/18f1ad1860ae43bab357d4a532fe6e07 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 19:34:02.844413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:02.845957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:02.859752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71becc0-7810-4203-a6a1-bc3b8a65135d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:02.920268 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:02.920939 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:02.921230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:02.921539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] PATH_INFO: `/v3/projects/5f9fdf5b71544afc99b57bd137b2e419` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:02.922605 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9fdf5b71544afc99b57bd137b2e419)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:02.924503 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:03.027720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9fdf5b71544afc99b57bd137b2e419', 'request_id': 'req-eab8328c-f8f6-4a83-8461-cbd293d10da2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:03.034391 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9fdf5b71544afc99b57bd137b2e419', 'request_id': 'req-eab8328c-f8f6-4a83-8461-cbd293d10da2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:03.041062 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9fdf5b71544afc99b57bd137b2e419', 'request_id': 'req-eab8328c-f8f6-4a83-8461-cbd293d10da2'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:03.114530 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] The token cache is being invalidate because project 5f9fdf5b71544afc99b57bd137b2e419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:34:03.114848 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:03.115484 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-eab8328c-f8f6-4a83-8461-cbd293d10da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:03.116985 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1419/2875] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:34:02 2022] DELETE /identity/v3/projects/5f9fdf5b71544afc99b57bd137b2e419 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:03.122314 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9011007-7df4-47f7-9ec9-c08b1d0cdc8e None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:03.122545 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9011007-7df4-47f7-9ec9-c08b1d0cdc8e None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:03.122824 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9011007-7df4-47f7-9ec9-c08b1d0cdc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:03.143697 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-f9011007-7df4-47f7-9ec9-c08b1d0cdc8e None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:03.145306 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f9011007-7df4-47f7-9ec9-c08b1d0cdc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:03.203708 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1457/2876] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:03.212918 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05847ce7-03c7-4d38-b933-c7231a621122 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:03.213231 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05847ce7-03c7-4d38-b933-c7231a621122 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:03.213513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05847ce7-03c7-4d38-b933-c7231a621122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:03.234170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-05847ce7-03c7-4d38-b933-c7231a621122 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:03.235454 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-05847ce7-03c7-4d38-b933-c7231a621122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:03.243743 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1420/2877] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:03.247809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9011007-7df4-47f7-9ec9-c08b1d0cdc8e None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:03.257449 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9011007-7df4-47f7-9ec9-c08b1d0cdc8e None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:03.261981 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960a5f0-134e-4767-8e91-11d57caa1dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:03.262579 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960a5f0-134e-4767-8e91-11d57caa1dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:03.262853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960a5f0-134e-4767-8e91-11d57caa1dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:03.263120 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960a5f0-134e-4767-8e91-11d57caa1dd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:03.263745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960a5f0-134e-4767-8e91-11d57caa1dd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 19:34:03.264042 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960a5f0-134e-4767-8e91-11d57caa1dd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:03.265932 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960a5f0-134e-4767-8e91-11d57caa1dd8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:03.272687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:03.277889 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1458/2878] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 19:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 19:34:03.293198 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05847ce7-03c7-4d38-b933-c7231a621122 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:03.302413 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05847ce7-03c7-4d38-b933-c7231a621122 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:03.307435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4bbcd-1bd4-43e1-b786-e97c78b93177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:03.316203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4bbcd-1bd4-43e1-b786-e97c78b93177 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:03.316203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4bbcd-1bd4-43e1-b786-e97c78b93177 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:03.316203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4bbcd-1bd4-43e1-b786-e97c78b93177 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:03.316203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4bbcd-1bd4-43e1-b786-e97c78b93177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:03.316203 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4bbcd-1bd4-43e1-b786-e97c78b93177 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:03.404776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1421/2879] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:34:03 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:34:03.407936 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960a5f0-134e-4767-8e91-11d57caa1dd8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:03.417911 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960a5f0-134e-4767-8e91-11d57caa1dd8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:03.473853 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbb064-58cc-4d21-97e6-a0c0c90b5120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:03.474376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbb064-58cc-4d21-97e6-a0c0c90b5120 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:03.474663 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbb064-58cc-4d21-97e6-a0c0c90b5120 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:03.474928 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbb064-58cc-4d21-97e6-a0c0c90b5120 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:03.475560 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbb064-58cc-4d21-97e6-a0c0c90b5120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:03.477615 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbb064-58cc-4d21-97e6-a0c0c90b5120 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:03.578349 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1459/2880] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:34:03 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:34:03.582415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4bbcd-1bd4-43e1-b786-e97c78b93177 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:03.595419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4bbcd-1bd4-43e1-b786-e97c78b93177 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:03.600264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad646e8-3b95-4b0d-bf84-8d6ff8569ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:03.601343 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad646e8-3b95-4b0d-bf84-8d6ff8569ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:03.601629 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad646e8-3b95-4b0d-bf84-8d6ff8569ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:03.601904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad646e8-3b95-4b0d-bf84-8d6ff8569ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:03.602781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad646e8-3b95-4b0d-bf84-8d6ff8569ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:03.604401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad646e8-3b95-4b0d-bf84-8d6ff8569ce1 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:03.614655 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1422/2881] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:03.618620 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbb064-58cc-4d21-97e6-a0c0c90b5120 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:03.628523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbb064-58cc-4d21-97e6-a0c0c90b5120 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:03.633034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1864a11-f02b-4178-afd3-50ab3f0e058f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:03.634284 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1864a11-f02b-4178-afd3-50ab3f0e058f admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:03.634563 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1864a11-f02b-4178-afd3-50ab3f0e058f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:03.634834 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1864a11-f02b-4178-afd3-50ab3f0e058f admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:03.635464 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1864a11-f02b-4178-afd3-50ab3f0e058f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:03.637021 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1864a11-f02b-4178-afd3-50ab3f0e058f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:03.647653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1460/2882] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 19:34:03.652404 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad646e8-3b95-4b0d-bf84-8d6ff8569ce1 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:03.663041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad646e8-3b95-4b0d-bf84-8d6ff8569ce1 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:03.668049 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ec8fc-fa66-4812-846c-df42e2ad1c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:03.668698 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ec8fc-fa66-4812-846c-df42e2ad1c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:03.668979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ec8fc-fa66-4812-846c-df42e2ad1c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:03.669292 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ec8fc-fa66-4812-846c-df42e2ad1c1d admin admin] PATH_INFO: `/v3/projects/a3d1f1b738f5490d992148171c87d338/users/fbf250d9c2ab4f2a96131c9137af627b/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:03.672329 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:03.683817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ec8fc-fa66-4812-846c-df42e2ad1c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d1f1b738f5490d992148171c87d338, user_id=fbf250d9c2ab4f2a96131c9137af627b, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:03.686506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ec8fc-fa66-4812-846c-df42e2ad1c1d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:03.781015 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1423/2883] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:03 2022] PUT /identity/v3/projects/a3d1f1b738f5490d992148171c87d338/users/fbf250d9c2ab4f2a96131c9137af627b/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:03.791301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3798b-e0ca-4789-83f9-9eb43e3f978d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:03.791590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3798b-e0ca-4789-83f9-9eb43e3f978d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:03.791869 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3798b-e0ca-4789-83f9-9eb43e3f978d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:03.820128 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-21c3798b-e0ca-4789-83f9-9eb43e3f978d None None] MFA Rules not processed for user `fbf250d9c2ab4f2a96131c9137af627b`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:03.822210 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-21c3798b-e0ca-4789-83f9-9eb43e3f978d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:03.886348 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1461/2884] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:03.890947 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ec8fc-fa66-4812-846c-df42e2ad1c1d admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:03.953306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ec8fc-fa66-4812-846c-df42e2ad1c1d admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:03.957935 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf72e29-1dd4-4f06-a8da-bb3dacf20c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:03.958533 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf72e29-1dd4-4f06-a8da-bb3dacf20c54 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:03.958810 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf72e29-1dd4-4f06-a8da-bb3dacf20c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:03.959081 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf72e29-1dd4-4f06-a8da-bb3dacf20c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:03.959699 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf72e29-1dd4-4f06-a8da-bb3dacf20c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:03.961620 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf72e29-1dd4-4f06-a8da-bb3dacf20c54 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:04.038085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1424/2885] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:34:03 2022] POST /identity/v3/projects => generated 393 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:34:04.042387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3798b-e0ca-4789-83f9-9eb43e3f978d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:04.053079 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3798b-e0ca-4789-83f9-9eb43e3f978d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:04.110086 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e265-4ffb-46a5-af94-976eef73f7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:04.111083 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e265-4ffb-46a5-af94-976eef73f7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:04.111587 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e265-4ffb-46a5-af94-976eef73f7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:04.112081 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9e265-4ffb-46a5-af94-976eef73f7b6 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:04.113171 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9e265-4ffb-46a5-af94-976eef73f7b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:04.116408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9e265-4ffb-46a5-af94-976eef73f7b6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:04.128475 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:04.203145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1462/2886] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:34:04 2022] POST /identity/v3/users => generated 303 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:34:04.207242 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf72e29-1dd4-4f06-a8da-bb3dacf20c54 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:04.217750 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf72e29-1dd4-4f06-a8da-bb3dacf20c54 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:04.222389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c49e09-2ca7-4486-ba7b-c2684c8f369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:04.223014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49e09-2ca7-4486-ba7b-c2684c8f369f admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:04.223297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49e09-2ca7-4486-ba7b-c2684c8f369f admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:04.223570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49e09-2ca7-4486-ba7b-c2684c8f369f admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:04.224377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c49e09-2ca7-4486-ba7b-c2684c8f369f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:04.225916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c49e09-2ca7-4486-ba7b-c2684c8f369f admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:04.237535 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1425/2887] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:04.241687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e265-4ffb-46a5-af94-976eef73f7b6 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:04.252344 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9e265-4ffb-46a5-af94-976eef73f7b6 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:04.257122 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a58adc-c009-44ce-93dd-dea44cd47936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:04.257775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a58adc-c009-44ce-93dd-dea44cd47936 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:04.258060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a58adc-c009-44ce-93dd-dea44cd47936 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:04.258343 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a58adc-c009-44ce-93dd-dea44cd47936 admin admin] PATH_INFO: `/v3/projects/4a52c45598704131a1f405cd1c434334/users/59b752e976d441aab44948b535182869/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:04.278885 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a58adc-c009-44ce-93dd-dea44cd47936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a52c45598704131a1f405cd1c434334, user_id=59b752e976d441aab44948b535182869, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:04.281848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a58adc-c009-44ce-93dd-dea44cd47936 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:04.334541 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1463/2888] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:04 2022] PUT /identity/v3/projects/4a52c45598704131a1f405cd1c434334/users/59b752e976d441aab44948b535182869/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:04.339105 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c49e09-2ca7-4486-ba7b-c2684c8f369f admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:04.349290 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c49e09-2ca7-4486-ba7b-c2684c8f369f admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:04.409101 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76a7ef-074f-4519-9206-e06b6a9f9467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:04.409748 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76a7ef-074f-4519-9206-e06b6a9f9467 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:04.410034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76a7ef-074f-4519-9206-e06b6a9f9467 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:04.410309 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76a7ef-074f-4519-9206-e06b6a9f9467 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:04.410951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76a7ef-074f-4519-9206-e06b6a9f9467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:04.412547 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76a7ef-074f-4519-9206-e06b6a9f9467 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:04.422859 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1426/2889] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:04.427200 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a58adc-c009-44ce-93dd-dea44cd47936 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:04.437841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a58adc-c009-44ce-93dd-dea44cd47936 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:04.442903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b123e78-3914-4e1e-8854-8f8cdff9addb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:04.454064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b123e78-3914-4e1e-8854-8f8cdff9addb admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:04.454064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b123e78-3914-4e1e-8854-8f8cdff9addb admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:04.454064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b123e78-3914-4e1e-8854-8f8cdff9addb admin admin] PATH_INFO: `/v3/projects/4a52c45598704131a1f405cd1c434334/users/59b752e976d441aab44948b535182869/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:04.454064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b123e78-3914-4e1e-8854-8f8cdff9addb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a52c45598704131a1f405cd1c434334, user_id=59b752e976d441aab44948b535182869, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:04.454064 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b123e78-3914-4e1e-8854-8f8cdff9addb admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:04.460354 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1464/2890] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:04 2022] PUT /identity/v3/projects/4a52c45598704131a1f405cd1c434334/users/59b752e976d441aab44948b535182869/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:04.472151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc618694-517a-4b24-8b41-2d5c5b2c23f2 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:04.472441 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc618694-517a-4b24-8b41-2d5c5b2c23f2 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:04.472718 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc618694-517a-4b24-8b41-2d5c5b2c23f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:04.500334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-bc618694-517a-4b24-8b41-2d5c5b2c23f2 None None] MFA Rules not processed for user `59b752e976d441aab44948b535182869`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:04.501616 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bc618694-517a-4b24-8b41-2d5c5b2c23f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:04.546982 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:04.571994 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1427/2891] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:04 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:04.582561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b123e78-3914-4e1e-8854-8f8cdff9addb admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:04.584290 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-9b123e78-3914-4e1e-8854-8f8cdff9addb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:04.643494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b123e78-3914-4e1e-8854-8f8cdff9addb admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:04.647337 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8e593-9840-4711-ab7b-fbb1b49a97fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:04.647943 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8e593-9840-4711-ab7b-fbb1b49a97fe service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:04.648223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8e593-9840-4711-ab7b-fbb1b49a97fe service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:04.648495 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8e593-9840-4711-ab7b-fbb1b49a97fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:04.657723 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8e593-9840-4711-ab7b-fbb1b49a97fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:04.659130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8e593-9840-4711-ab7b-fbb1b49a97fe service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:04.668801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1465/2892] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:04 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:04.715415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc618694-517a-4b24-8b41-2d5c5b2c23f2 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:04.716751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bc618694-517a-4b24-8b41-2d5c5b2c23f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:04.728904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc618694-517a-4b24-8b41-2d5c5b2c23f2 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:04.732820 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45129487-9cbf-4cdc-9e0a-8346bb312276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:04.733467 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129487-9cbf-4cdc-9e0a-8346bb312276 service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:04.733749 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129487-9cbf-4cdc-9e0a-8346bb312276 service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:04.734024 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129487-9cbf-4cdc-9e0a-8346bb312276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:04.791833 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45129487-9cbf-4cdc-9e0a-8346bb312276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:04.793300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45129487-9cbf-4cdc-9e0a-8346bb312276 service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:04.802111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1428/2893] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:09.429238 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8e593-9840-4711-ab7b-fbb1b49a97fe service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:09.490801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8e593-9840-4711-ab7b-fbb1b49a97fe service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:09.495353 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:09.495971 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:09.496251 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:09.496524 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] PATH_INFO: `/v3/users/fbf250d9c2ab4f2a96131c9137af627b` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:09.497640 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbf250d9c2ab4f2a96131c9137af627b)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:09.499526 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:09.640275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbf250d9c2ab4f2a96131c9137af627b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:09.640920 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbf250d9c2ab4f2a96131c9137af627b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:09.641724 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbf250d9c2ab4f2a96131c9137af627b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:09.651531 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbf250d9c2ab4f2a96131c9137af627b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:09.657731 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:09.664809 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbf250d9c2ab4f2a96131c9137af627b'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:09.767802 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1466/2894] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:34:09 2022] DELETE /identity/v3/users/fbf250d9c2ab4f2a96131c9137af627b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:09.772247 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52eb33-22ed-4613-8966-c148c99034c3 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:09.772707 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52eb33-22ed-4613-8966-c148c99034c3 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:09.772883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52eb33-22ed-4613-8966-c148c99034c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:09.792663 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-8f52eb33-22ed-4613-8966-c148c99034c3 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:09.793943 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8f52eb33-22ed-4613-8966-c148c99034c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:09.857294 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1429/2895] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:09.866538 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:09.867974 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:09.928105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4e1fe-2737-42ef-8562-bad0e5968d79 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:09.931760 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6184c50-5e01-4c5a-a81c-a3bf41605c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:09.932460 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6184c50-5e01-4c5a-a81c-a3bf41605c98 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:09.932871 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6184c50-5e01-4c5a-a81c-a3bf41605c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:09.933074 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6184c50-5e01-4c5a-a81c-a3bf41605c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:09.941984 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6184c50-5e01-4c5a-a81c-a3bf41605c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:09.943398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6184c50-5e01-4c5a-a81c-a3bf41605c98 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:09.953112 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1467/2896] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:10.364922 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52eb33-22ed-4613-8966-c148c99034c3 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:10.366386 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-8f52eb33-22ed-4613-8966-c148c99034c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:10.379369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52eb33-22ed-4613-8966-c148c99034c3 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:10.393262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:10.393961 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:10.394243 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:10.394574 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] PATH_INFO: `/v3/projects/a3d1f1b738f5490d992148171c87d338` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:10.395780 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d1f1b738f5490d992148171c87d338)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:10.397773 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:10.567502 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d1f1b738f5490d992148171c87d338', 'request_id': 'req-832aac76-aad8-43c1-91dc-835c50ca9088'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:10.574283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d1f1b738f5490d992148171c87d338', 'request_id': 'req-832aac76-aad8-43c1-91dc-835c50ca9088'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:10.579854 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d1f1b738f5490d992148171c87d338', 'request_id': 'req-832aac76-aad8-43c1-91dc-835c50ca9088'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:10.658162 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] The token cache is being invalidate because project a3d1f1b738f5490d992148171c87d338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:34:10.658440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:10.659076 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:10.660667 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1430/2897] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:34:10 2022] DELETE /identity/v3/projects/a3d1f1b738f5490d992148171c87d338 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:10.664664 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6184c50-5e01-4c5a-a81c-a3bf41605c98 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:10.674375 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6184c50-5e01-4c5a-a81c-a3bf41605c98 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:10.731357 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:10.731988 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:10.732269 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:10.732543 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] PATH_INFO: `/v3/users/59b752e976d441aab44948b535182869` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:10.733680 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b752e976d441aab44948b535182869)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:10.735625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:10.873373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b752e976d441aab44948b535182869'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:10.873961 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b752e976d441aab44948b535182869'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:10.874600 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b752e976d441aab44948b535182869'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:10.881698 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b752e976d441aab44948b535182869'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:10.894967 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-29175cd7-b570-4f4f-b1d5-ba26aa44e3a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b752e976d441aab44948b535182869'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:10.983110 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1468/2898] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:34:10 2022] DELETE /identity/v3/users/59b752e976d441aab44948b535182869 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:11.371525 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:11.373311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:11.383781 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:11.390232 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832aac76-aad8-43c1-91dc-835c50ca9088 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:11.450091 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:11.450728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:11.451013 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:11.451288 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] PATH_INFO: `/v3/projects/4a52c45598704131a1f405cd1c434334` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:11.452419 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a52c45598704131a1f405cd1c434334)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:11.454355 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:11.575207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a52c45598704131a1f405cd1c434334', 'request_id': 'req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:11.581377 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a52c45598704131a1f405cd1c434334', 'request_id': 'req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:11.586955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a52c45598704131a1f405cd1c434334', 'request_id': 'req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:11.647687 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] The token cache is being invalidate because project 4a52c45598704131a1f405cd1c434334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:34:11.647920 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:11.648562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-0fb2b26c-6043-41b5-ae84-fb16c9a1f450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:11.650150 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1431/2899] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:34:11 2022] DELETE /identity/v3/projects/4a52c45598704131a1f405cd1c434334 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:11.655711 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cf0c9-993c-4c9a-8d20-fdc2fd51517f None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:11.655997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cf0c9-993c-4c9a-8d20-fdc2fd51517f None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:11.656275 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cf0c9-993c-4c9a-8d20-fdc2fd51517f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:11.664898 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:11.683034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-608cf0c9-993c-4c9a-8d20-fdc2fd51517f None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:11.684260 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-608cf0c9-993c-4c9a-8d20-fdc2fd51517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:11.750331 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1469/2900] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:11.758034 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139058-e744-4ad6-b0f9-7e44c7636aab None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:11.758323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139058-e744-4ad6-b0f9-7e44c7636aab None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:11.758599 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2139058-e744-4ad6-b0f9-7e44c7636aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:11.780427 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-e2139058-e744-4ad6-b0f9-7e44c7636aab None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:11.781682 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e2139058-e744-4ad6-b0f9-7e44c7636aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:11.789627 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1432/2901] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:11.794490 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cf0c9-993c-4c9a-8d20-fdc2fd51517f None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:11.804735 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cf0c9-993c-4c9a-8d20-fdc2fd51517f None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:11.810041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99275c86-c3ee-49d4-bb44-bfc73e36fbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:11.810642 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275c86-c3ee-49d4-bb44-bfc73e36fbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:11.810923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275c86-c3ee-49d4-bb44-bfc73e36fbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:11.811198 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275c86-c3ee-49d4-bb44-bfc73e36fbe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:11.811841 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99275c86-c3ee-49d4-bb44-bfc73e36fbe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 19:34:11.812146 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99275c86-c3ee-49d4-bb44-bfc73e36fbe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:11.813775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99275c86-c3ee-49d4-bb44-bfc73e36fbe4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:11.825218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1470/2902] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 19:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 19:34:11.829492 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2139058-e744-4ad6-b0f9-7e44c7636aab None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:11.838621 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2139058-e744-4ad6-b0f9-7e44c7636aab None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:11.843202 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836ec34-9977-49d1-8d12-c573172ab7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:11.843788 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836ec34-9977-49d1-8d12-c573172ab7ff admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:11.844066 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836ec34-9977-49d1-8d12-c573172ab7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:11.844334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836ec34-9977-49d1-8d12-c573172ab7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:11.844954 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836ec34-9977-49d1-8d12-c573172ab7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:11.846858 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836ec34-9977-49d1-8d12-c573172ab7ff admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:11.927831 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1433/2903] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:34:11 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:34:11.932774 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99275c86-c3ee-49d4-bb44-bfc73e36fbe4 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:11.943211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99275c86-c3ee-49d4-bb44-bfc73e36fbe4 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:11.999881 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa23b9-553e-4205-a1e2-2d3927360097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:12.000489 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa23b9-553e-4205-a1e2-2d3927360097 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:12.000768 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa23b9-553e-4205-a1e2-2d3927360097 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:12.001071 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa23b9-553e-4205-a1e2-2d3927360097 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:12.001705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa23b9-553e-4205-a1e2-2d3927360097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:12.003606 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa23b9-553e-4205-a1e2-2d3927360097 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:12.127494 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1471/2904] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:34:11 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:34:12.131608 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836ec34-9977-49d1-8d12-c573172ab7ff admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:12.142140 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836ec34-9977-49d1-8d12-c573172ab7ff admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:12.147227 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6654d-bd68-4cb3-a2be-50cda6e12033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:12.147827 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6654d-bd68-4cb3-a2be-50cda6e12033 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:12.148109 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6654d-bd68-4cb3-a2be-50cda6e12033 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:12.148409 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6654d-bd68-4cb3-a2be-50cda6e12033 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:12.149071 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6654d-bd68-4cb3-a2be-50cda6e12033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:12.151222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6654d-bd68-4cb3-a2be-50cda6e12033 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:12.161553 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1434/2905] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:12.165335 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa23b9-553e-4205-a1e2-2d3927360097 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:12.176215 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa23b9-553e-4205-a1e2-2d3927360097 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:12.180876 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6fca7-8fd4-4ac8-a74f-8e1b0932e160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:12.181446 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6fca7-8fd4-4ac8-a74f-8e1b0932e160 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:12.181904 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6fca7-8fd4-4ac8-a74f-8e1b0932e160 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:12.182205 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6fca7-8fd4-4ac8-a74f-8e1b0932e160 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:12.182815 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6fca7-8fd4-4ac8-a74f-8e1b0932e160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:12.184376 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6fca7-8fd4-4ac8-a74f-8e1b0932e160 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:12.194982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1472/2906] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:12.198907 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6654d-bd68-4cb3-a2be-50cda6e12033 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:12.209016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6654d-bd68-4cb3-a2be-50cda6e12033 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:12.214222 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e0484-a295-4bf4-a56d-f38db83f6508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:12.214862 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e0484-a295-4bf4-a56d-f38db83f6508 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:12.215145 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e0484-a295-4bf4-a56d-f38db83f6508 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:12.215421 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e0484-a295-4bf4-a56d-f38db83f6508 admin admin] PATH_INFO: `/v3/projects/8229c0bc9b7f4a1e81faf0740b5dff37/users/8233f7af6168495094939ecf4b2267de/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:12.230523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e0484-a295-4bf4-a56d-f38db83f6508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8229c0bc9b7f4a1e81faf0740b5dff37, user_id=8233f7af6168495094939ecf4b2267de, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:12.233300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e0484-a295-4bf4-a56d-f38db83f6508 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:12.298529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1435/2907] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:12 2022] PUT /identity/v3/projects/8229c0bc9b7f4a1e81faf0740b5dff37/users/8233f7af6168495094939ecf4b2267de/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:12.308687 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222702d-4bdb-4c95-adf1-98dbfbf091cb None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:12.310411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222702d-4bdb-4c95-adf1-98dbfbf091cb None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:12.310411 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222702d-4bdb-4c95-adf1-98dbfbf091cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:12.342060 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-4222702d-4bdb-4c95-adf1-98dbfbf091cb None None] MFA Rules not processed for user `8233f7af6168495094939ecf4b2267de`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:12.343278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4222702d-4bdb-4c95-adf1-98dbfbf091cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:12.405262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:12.410326 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1473/2908] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:12.414963 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e0484-a295-4bf4-a56d-f38db83f6508 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:12.448976 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:12.479636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e0484-a295-4bf4-a56d-f38db83f6508 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:12.484359 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df129a58-187a-4521-8668-6e2b37cfff04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:12.484955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df129a58-187a-4521-8668-6e2b37cfff04 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:12.485263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df129a58-187a-4521-8668-6e2b37cfff04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:12.485536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df129a58-187a-4521-8668-6e2b37cfff04 admin admin] PATH_INFO: `/v3/projects` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:12.486152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df129a58-187a-4521-8668-6e2b37cfff04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:12.488124 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df129a58-187a-4521-8668-6e2b37cfff04 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:12.581334 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1436/2909] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:34:12 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:34:12.585508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222702d-4bdb-4c95-adf1-98dbfbf091cb None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:12.597154 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222702d-4bdb-4c95-adf1-98dbfbf091cb None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:12.656259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5478c-9225-486b-9c14-f03659462c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:12.656870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5478c-9225-486b-9c14-f03659462c94 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:12.657402 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5478c-9225-486b-9c14-f03659462c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:12.657604 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5478c-9225-486b-9c14-f03659462c94 admin admin] PATH_INFO: `/v3/users` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:12.658255 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a5478c-9225-486b-9c14-f03659462c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:12.660185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a5478c-9225-486b-9c14-f03659462c94 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:12.734412 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1474/2910] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:34:12 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:34:12.738510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df129a58-187a-4521-8668-6e2b37cfff04 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:12.749726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df129a58-187a-4521-8668-6e2b37cfff04 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:12.754254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427375f-1d25-4e18-80f3-8e88800d6e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:12.754846 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427375f-1d25-4e18-80f3-8e88800d6e1a admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:12.755123 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427375f-1d25-4e18-80f3-8e88800d6e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:12.755394 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427375f-1d25-4e18-80f3-8e88800d6e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:12.756027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427375f-1d25-4e18-80f3-8e88800d6e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:12.757641 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427375f-1d25-4e18-80f3-8e88800d6e1a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:12.768130 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1437/2911] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:12.772262 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5478c-9225-486b-9c14-f03659462c94 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:12.783044 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5478c-9225-486b-9c14-f03659462c94 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:12.787706 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61aa1d-45bc-410f-9a9c-d87a2474cfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:12.788345 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61aa1d-45bc-410f-9a9c-d87a2474cfe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:12.788624 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61aa1d-45bc-410f-9a9c-d87a2474cfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:12.788903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61aa1d-45bc-410f-9a9c-d87a2474cfe7 admin admin] PATH_INFO: `/v3/projects/75869ff19fa440c0b8c027c882aad4ba/users/2bd26943b8e645eb88377e852984d035/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:12.803847 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61aa1d-45bc-410f-9a9c-d87a2474cfe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75869ff19fa440c0b8c027c882aad4ba, user_id=2bd26943b8e645eb88377e852984d035, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:12.807039 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61aa1d-45bc-410f-9a9c-d87a2474cfe7 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:12.910041 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1475/2912] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:12 2022] PUT /identity/v3/projects/75869ff19fa440c0b8c027c882aad4ba/users/2bd26943b8e645eb88377e852984d035/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:12.914053 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427375f-1d25-4e18-80f3-8e88800d6e1a admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:12.925002 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427375f-1d25-4e18-80f3-8e88800d6e1a admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:12.981738 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3af12-fe4d-439a-ad14-fef2469a1c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:12.982332 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3af12-fe4d-439a-ad14-fef2469a1c8a admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:12.982610 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3af12-fe4d-439a-ad14-fef2469a1c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:12.982880 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3af12-fe4d-439a-ad14-fef2469a1c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:12.983508 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3af12-fe4d-439a-ad14-fef2469a1c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:12.985151 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3af12-fe4d-439a-ad14-fef2469a1c8a admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:12.995398 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1438/2913] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:12.999315 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61aa1d-45bc-410f-9a9c-d87a2474cfe7 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:13.009388 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61aa1d-45bc-410f-9a9c-d87a2474cfe7 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:13.014423 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b474e30-88ec-45a1-a553-442de611c300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:13.015063 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b474e30-88ec-45a1-a553-442de611c300 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:13.015340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b474e30-88ec-45a1-a553-442de611c300 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:13.015610 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b474e30-88ec-45a1-a553-442de611c300 admin admin] PATH_INFO: `/v3/projects/75869ff19fa440c0b8c027c882aad4ba/users/2bd26943b8e645eb88377e852984d035/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:13.017528 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b474e30-88ec-45a1-a553-442de611c300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75869ff19fa440c0b8c027c882aad4ba, user_id=2bd26943b8e645eb88377e852984d035, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:13.020139 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b474e30-88ec-45a1-a553-442de611c300 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:13.031507 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1476/2914] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:12 2022] PUT /identity/v3/projects/75869ff19fa440c0b8c027c882aad4ba/users/2bd26943b8e645eb88377e852984d035/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:13.106729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e355c63-1bd5-4e82-88de-4fc90978291a None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:13.107042 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e355c63-1bd5-4e82-88de-4fc90978291a None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:13.107321 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e355c63-1bd5-4e82-88de-4fc90978291a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:13.135904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-9e355c63-1bd5-4e82-88de-4fc90978291a None None] MFA Rules not processed for user `2bd26943b8e645eb88377e852984d035`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:13.137164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-9e355c63-1bd5-4e82-88de-4fc90978291a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:13.204384 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1439/2915] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 19:34:13.215938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b474e30-88ec-45a1-a553-442de611c300 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:13.217688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-4b474e30-88ec-45a1-a553-442de611c300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:13.276775 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b474e30-88ec-45a1-a553-442de611c300 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:13.280670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42abac-631b-4978-b569-182cd5b8b4ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:13.281300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42abac-631b-4978-b569-182cd5b8b4ab service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:13.281573 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42abac-631b-4978-b569-182cd5b8b4ab service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:13.281843 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42abac-631b-4978-b569-182cd5b8b4ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:13.290590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42abac-631b-4978-b569-182cd5b8b4ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:13.292033 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42abac-631b-4978-b569-182cd5b8b4ab service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:13.301650 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1477/2916] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:13.441373 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e355c63-1bd5-4e82-88de-4fc90978291a None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:13.469010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:13.503832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e355c63-1bd5-4e82-88de-4fc90978291a None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:13.508578 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:13.509241 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:13.509585 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:13.510483 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] PATH_INFO: `/v3/users/8233f7af6168495094939ecf4b2267de` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:13.511495 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8233f7af6168495094939ecf4b2267de)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:13.513447 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:13.663010 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8233f7af6168495094939ecf4b2267de'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:13.663652 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8233f7af6168495094939ecf4b2267de'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:13.664252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8233f7af6168495094939ecf4b2267de'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:13.677744 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8233f7af6168495094939ecf4b2267de'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:13.685440 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8233f7af6168495094939ecf4b2267de'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:13.739813 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1440/2917] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:34:13 2022] DELETE /identity/v3/users/8233f7af6168495094939ecf4b2267de => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:13.744840 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb28e06-7c6a-476a-9478-4a9aa9aa28e5 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:13.745389 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb28e06-7c6a-476a-9478-4a9aa9aa28e5 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:13.745675 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb28e06-7c6a-476a-9478-4a9aa9aa28e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:13.765382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-dbb28e06-7c6a-476a-9478-4a9aa9aa28e5 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:13.766523 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-dbb28e06-7c6a-476a-9478-4a9aa9aa28e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:13.813470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:13.826307 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1478/2918] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:13.835942 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:13.837375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:13.895646 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343274d-1d43-4f3b-ac9f-cb3f6cea55ba admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:13.899297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a05be-a6ac-4b23-8c50-f85f7a624ef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:13.899904 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a05be-a6ac-4b23-8c50-f85f7a624ef7 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:13.900186 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a05be-a6ac-4b23-8c50-f85f7a624ef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:13.901271 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a05be-a6ac-4b23-8c50-f85f7a624ef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:13.910486 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a05be-a6ac-4b23-8c50-f85f7a624ef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:13.911916 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a05be-a6ac-4b23-8c50-f85f7a624ef7 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:13.921439 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1441/2919] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:14.358150 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb28e06-7c6a-476a-9478-4a9aa9aa28e5 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:14.359612 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-dbb28e06-7c6a-476a-9478-4a9aa9aa28e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:14.371874 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb28e06-7c6a-476a-9478-4a9aa9aa28e5 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:14.376554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:14.377270 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:14.377546 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:14.384703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] PATH_INFO: `/v3/projects/8229c0bc9b7f4a1e81faf0740b5dff37` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:14.384703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8229c0bc9b7f4a1e81faf0740b5dff37)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:14.384703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:14.521550 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8229c0bc9b7f4a1e81faf0740b5dff37', 'request_id': 'req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:14.528479 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8229c0bc9b7f4a1e81faf0740b5dff37', 'request_id': 'req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:14.600592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8229c0bc9b7f4a1e81faf0740b5dff37', 'request_id': 'req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:14.612634 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] The token cache is being invalidate because project 8229c0bc9b7f4a1e81faf0740b5dff37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:34:14.613088 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:14.613754 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:14.615278 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1479/2920] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:34:14 2022] DELETE /identity/v3/projects/8229c0bc9b7f4a1e81faf0740b5dff37 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:14.619358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a05be-a6ac-4b23-8c50-f85f7a624ef7 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:14.629106 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a05be-a6ac-4b23-8c50-f85f7a624ef7 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:14.673957 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:14.686878 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:14.687509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:14.687797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:14.688080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] PATH_INFO: `/v3/users/2bd26943b8e645eb88377e852984d035` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:14.689233 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd26943b8e645eb88377e852984d035)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:14.691168 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:14.892686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd26943b8e645eb88377e852984d035'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:14.893370 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd26943b8e645eb88377e852984d035'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:14.894058 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd26943b8e645eb88377e852984d035'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:14.908543 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd26943b8e645eb88377e852984d035'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:14.916041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-07f6950d-42b8-473c-b1f5-97ac906faf41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd26943b8e645eb88377e852984d035'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:14.989331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1442/2921] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:34:14 2022] DELETE /identity/v3/users/2bd26943b8e645eb88377e852984d035 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:15.710616 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:15.714145 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:15.726980 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8492772-4a7a-41cc-8070-9a2d13cf1fb1 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:15.790937 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:15.791569 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:15.791909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:15.792684 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] PATH_INFO: `/v3/projects/75869ff19fa440c0b8c027c882aad4ba` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:15.793861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] RBAC: Authorizing `identity:delete_project(project_id=75869ff19fa440c0b8c027c882aad4ba)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:15.795733 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:15.918839 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75869ff19fa440c0b8c027c882aad4ba', 'request_id': 'req-e455dc8e-b0fb-43d9-ac55-c2aad215196f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:15.925023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75869ff19fa440c0b8c027c882aad4ba', 'request_id': 'req-e455dc8e-b0fb-43d9-ac55-c2aad215196f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:15.979391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75869ff19fa440c0b8c027c882aad4ba', 'request_id': 'req-e455dc8e-b0fb-43d9-ac55-c2aad215196f'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:15.990863 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] The token cache is being invalidate because project 75869ff19fa440c0b8c027c882aad4ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:34:15.991212 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:15.991867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-e455dc8e-b0fb-43d9-ac55-c2aad215196f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:15.993429 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1480/2922] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:34:15 2022] DELETE /identity/v3/projects/75869ff19fa440c0b8c027c882aad4ba => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:15.998500 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685c65-9087-47c5-a6cd-3b2b50a7d532 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:15.998955 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685c65-9087-47c5-a6cd-3b2b50a7d532 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:15.999063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf685c65-9087-47c5-a6cd-3b2b50a7d532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.018883 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-bf685c65-9087-47c5-a6cd-3b2b50a7d532 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:16.020079 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-bf685c65-9087-47c5-a6cd-3b2b50a7d532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:16.078637 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1443/2923] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 19:34:16.086130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa491b-dc61-4ad2-845e-bbdfa627f8f1 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:16.086424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa491b-dc61-4ad2-845e-bbdfa627f8f1 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:16.086717 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa491b-dc61-4ad2-845e-bbdfa627f8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.102590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:16.108859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-45aa491b-dc61-4ad2-845e-bbdfa627f8f1 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:16.110105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-45aa491b-dc61-4ad2-845e-bbdfa627f8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:16.117992 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1481/2924] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:16.122901 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf685c65-9087-47c5-a6cd-3b2b50a7d532 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:16.132851 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf685c65-9087-47c5-a6cd-3b2b50a7d532 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:16.138383 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385d2e4-5228-409e-a49e-80cc5aa1ae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:16.138979 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385d2e4-5228-409e-a49e-80cc5aa1ae60 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:16.141850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385d2e4-5228-409e-a49e-80cc5aa1ae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:16.141850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385d2e4-5228-409e-a49e-80cc5aa1ae60 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.141850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385d2e4-5228-409e-a49e-80cc5aa1ae60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 19:34:16.141850 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385d2e4-5228-409e-a49e-80cc5aa1ae60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:16.142161 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385d2e4-5228-409e-a49e-80cc5aa1ae60 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:16.155027 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1444/2925] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 19:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 19:34:16.159382 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa491b-dc61-4ad2-845e-bbdfa627f8f1 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:16.169320 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa491b-dc61-4ad2-845e-bbdfa627f8f1 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:16.173998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64fc02-42b4-4e2b-912a-511afc1ff676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:16.176281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64fc02-42b4-4e2b-912a-511afc1ff676 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:16.176281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64fc02-42b4-4e2b-912a-511afc1ff676 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:16.176281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64fc02-42b4-4e2b-912a-511afc1ff676 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.176281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64fc02-42b4-4e2b-912a-511afc1ff676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:16.177636 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64fc02-42b4-4e2b-912a-511afc1ff676 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:16.249648 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1482/2926] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:34:16 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:34:16.253724 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385d2e4-5228-409e-a49e-80cc5aa1ae60 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:16.265691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385d2e4-5228-409e-a49e-80cc5aa1ae60 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:16.313477 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:16.322192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e55abf-3c3d-4985-8d71-c794f8bb6383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:16.322795 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e55abf-3c3d-4985-8d71-c794f8bb6383 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:16.323635 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e55abf-3c3d-4985-8d71-c794f8bb6383 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:16.323938 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e55abf-3c3d-4985-8d71-c794f8bb6383 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.324577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e55abf-3c3d-4985-8d71-c794f8bb6383 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:16.326539 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e55abf-3c3d-4985-8d71-c794f8bb6383 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:16.409110 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1445/2927] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:34:16 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:34:16.413189 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64fc02-42b4-4e2b-912a-511afc1ff676 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:16.423584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64fc02-42b4-4e2b-912a-511afc1ff676 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:16.428266 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe10d38-d87e-4925-b2ed-5357b9dc109b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:16.428861 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe10d38-d87e-4925-b2ed-5357b9dc109b admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:16.429185 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe10d38-d87e-4925-b2ed-5357b9dc109b admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:16.429496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe10d38-d87e-4925-b2ed-5357b9dc109b admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.430101 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe10d38-d87e-4925-b2ed-5357b9dc109b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:16.431689 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe10d38-d87e-4925-b2ed-5357b9dc109b admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:16.441894 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1483/2928] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:16.445729 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e55abf-3c3d-4985-8d71-c794f8bb6383 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:16.456250 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e55abf-3c3d-4985-8d71-c794f8bb6383 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:16.460892 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4440ffa-1a0d-4d77-95dd-44b34719f685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:16.461523 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4440ffa-1a0d-4d77-95dd-44b34719f685 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:16.461809 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4440ffa-1a0d-4d77-95dd-44b34719f685 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:16.462087 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4440ffa-1a0d-4d77-95dd-44b34719f685 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.462728 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4440ffa-1a0d-4d77-95dd-44b34719f685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:16.465159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4440ffa-1a0d-4d77-95dd-44b34719f685 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:16.475767 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1446/2929] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:16.479941 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe10d38-d87e-4925-b2ed-5357b9dc109b admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:16.490258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe10d38-d87e-4925-b2ed-5357b9dc109b admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:16.495562 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89a0d4-b905-4baf-adff-7da9ff196d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:16.496846 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89a0d4-b905-4baf-adff-7da9ff196d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:16.497152 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89a0d4-b905-4baf-adff-7da9ff196d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:16.497436 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89a0d4-b905-4baf-adff-7da9ff196d03 admin admin] PATH_INFO: `/v3/projects/9f520f7d1f554ccaaecca645b648129a/users/3a98ba22d17c4abcaed30cd5eb32cb5e/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.513424 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89a0d4-b905-4baf-adff-7da9ff196d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f520f7d1f554ccaaecca645b648129a, user_id=3a98ba22d17c4abcaed30cd5eb32cb5e, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:16.516053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89a0d4-b905-4baf-adff-7da9ff196d03 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:16.607826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1484/2930] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:16 2022] PUT /identity/v3/projects/9f520f7d1f554ccaaecca645b648129a/users/3a98ba22d17c4abcaed30cd5eb32cb5e/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:16.617405 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dabafa-c790-4f58-a15f-6b57ffb9cbed None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:16.617691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dabafa-c790-4f58-a15f-6b57ffb9cbed None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:16.617984 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dabafa-c790-4f58-a15f-6b57ffb9cbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.646645 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-92dabafa-c790-4f58-a15f-6b57ffb9cbed None None] MFA Rules not processed for user `3a98ba22d17c4abcaed30cd5eb32cb5e`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:16.647843 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-92dabafa-c790-4f58-a15f-6b57ffb9cbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:16.712273 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1447/2931] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:16.717304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89a0d4-b905-4baf-adff-7da9ff196d03 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:16.779347 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89a0d4-b905-4baf-adff-7da9ff196d03 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:16.784218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908d4f1-60af-4491-a113-63fddbc43f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:16.784817 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908d4f1-60af-4491-a113-63fddbc43f4a admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:16.785130 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908d4f1-60af-4491-a113-63fddbc43f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:16.785405 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908d4f1-60af-4491-a113-63fddbc43f4a admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.786036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908d4f1-60af-4491-a113-63fddbc43f4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:16.787915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908d4f1-60af-4491-a113-63fddbc43f4a admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:16.918271 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1485/2932] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:34:16 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:34:16.922611 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dabafa-c790-4f58-a15f-6b57ffb9cbed None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:16.934164 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dabafa-c790-4f58-a15f-6b57ffb9cbed None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:16.989554 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a19c9c-b203-4ba3-87ae-ea387c1af138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:16.990152 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a19c9c-b203-4ba3-87ae-ea387c1af138 admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:16.990435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a19c9c-b203-4ba3-87ae-ea387c1af138 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:16.990708 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a19c9c-b203-4ba3-87ae-ea387c1af138 admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:16.991328 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a19c9c-b203-4ba3-87ae-ea387c1af138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:16.993306 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a19c9c-b203-4ba3-87ae-ea387c1af138 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:17.115226 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1448/2933] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:34:16 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:34:17.119439 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908d4f1-60af-4491-a113-63fddbc43f4a admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:17.130373 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908d4f1-60af-4491-a113-63fddbc43f4a admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:17.135057 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db604cc3-c224-4715-888d-057025f992c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:17.135763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db604cc3-c224-4715-888d-057025f992c8 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:17.136065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db604cc3-c224-4715-888d-057025f992c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:17.136470 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db604cc3-c224-4715-888d-057025f992c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:17.137143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db604cc3-c224-4715-888d-057025f992c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:17.138721 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db604cc3-c224-4715-888d-057025f992c8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:17.149106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1486/2934] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:17.153252 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a19c9c-b203-4ba3-87ae-ea387c1af138 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:17.163427 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a19c9c-b203-4ba3-87ae-ea387c1af138 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:17.168577 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fddf8-61e7-463d-b538-473a707936b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:17.169254 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fddf8-61e7-463d-b538-473a707936b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:17.169532 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fddf8-61e7-463d-b538-473a707936b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:17.169803 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fddf8-61e7-463d-b538-473a707936b2 admin admin] PATH_INFO: `/v3/projects/df6193c170c14c81b473fdbb1fc52bca/users/15dfd997884e49e2b89481e3af6c2374/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:17.184826 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9fddf8-61e7-463d-b538-473a707936b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6193c170c14c81b473fdbb1fc52bca, user_id=15dfd997884e49e2b89481e3af6c2374, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:17.187506 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9fddf8-61e7-463d-b538-473a707936b2 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:17.246837 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1449/2935] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:17 2022] PUT /identity/v3/projects/df6193c170c14c81b473fdbb1fc52bca/users/15dfd997884e49e2b89481e3af6c2374/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:17.250764 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db604cc3-c224-4715-888d-057025f992c8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:17.260763 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db604cc3-c224-4715-888d-057025f992c8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:17.318552 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e337b3e-4f08-47e3-9725-2caa7a9d31a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:17.319158 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e337b3e-4f08-47e3-9725-2caa7a9d31a4 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:17.319442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e337b3e-4f08-47e3-9725-2caa7a9d31a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:17.319719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e337b3e-4f08-47e3-9725-2caa7a9d31a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:17.320355 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e337b3e-4f08-47e3-9725-2caa7a9d31a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:17.321998 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e337b3e-4f08-47e3-9725-2caa7a9d31a4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:17.331879 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1487/2936] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:17.336369 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fddf8-61e7-463d-b538-473a707936b2 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:17.348096 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fddf8-61e7-463d-b538-473a707936b2 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:17.353063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64740b9-6041-4cce-8661-4c64cb158c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:17.353721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64740b9-6041-4cce-8661-4c64cb158c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:17.354004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64740b9-6041-4cce-8661-4c64cb158c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:17.354289 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64740b9-6041-4cce-8661-4c64cb158c49 admin admin] PATH_INFO: `/v3/projects/df6193c170c14c81b473fdbb1fc52bca/users/15dfd997884e49e2b89481e3af6c2374/roles/5c777de40b8449329d714d8c2229115e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:17.356415 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64740b9-6041-4cce-8661-4c64cb158c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6193c170c14c81b473fdbb1fc52bca, user_id=15dfd997884e49e2b89481e3af6c2374, role_id=5c777de40b8449329d714d8c2229115e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:17.359064 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64740b9-6041-4cce-8661-4c64cb158c49 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:17.369353 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1450/2937] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:17 2022] PUT /identity/v3/projects/df6193c170c14c81b473fdbb1fc52bca/users/15dfd997884e49e2b89481e3af6c2374/roles/5c777de40b8449329d714d8c2229115e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:17.380906 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6216-8cf9-4405-84bd-e801190ad963 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:17.384339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6216-8cf9-4405-84bd-e801190ad963 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:17.384339 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c6216-8cf9-4405-84bd-e801190ad963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:17.413011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:17.415918 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-c32c6216-8cf9-4405-84bd-e801190ad963 None None] MFA Rules not processed for user `15dfd997884e49e2b89481e3af6c2374`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:17.417293 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-c32c6216-8cf9-4405-84bd-e801190ad963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:17.486867 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1488/2938] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:17.499752 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64740b9-6041-4cce-8661-4c64cb158c49 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:17.501513 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-e64740b9-6041-4cce-8661-4c64cb158c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:17.516735 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:17.560444 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64740b9-6041-4cce-8661-4c64cb158c49 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:17.564261 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecd72b-9b86-4f53-88a4-eb2649db9fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:17.564870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd72b-9b86-4f53-88a4-eb2649db9fbd service cinder] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:17.565184 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd72b-9b86-4f53-88a4-eb2649db9fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:17.565463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd72b-9b86-4f53-88a4-eb2649db9fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:17.576041 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecd72b-9b86-4f53-88a4-eb2649db9fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:17.577572 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecd72b-9b86-4f53-88a4-eb2649db9fbd service cinder] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:17.587260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1451/2939] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:17.729143 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c6216-8cf9-4405-84bd-e801190ad963 None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:17.739432 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c6216-8cf9-4405-84bd-e801190ad963 None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:17.743106 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ddca6-ef24-4e35-8c32-07b814f40386 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:17.743703 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ddca6-ef24-4e35-8c32-07b814f40386 service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:17.743982 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ddca6-ef24-4e35-8c32-07b814f40386 service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:17.744253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ddca6-ef24-4e35-8c32-07b814f40386 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:17.801301 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ddca6-ef24-4e35-8c32-07b814f40386 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:17.802743 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ddca6-ef24-4e35-8c32-07b814f40386 service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:17.812741 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1489/2940] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:18.302793 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecd72b-9b86-4f53-88a4-eb2649db9fbd service cinder] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:18.370908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecd72b-9b86-4f53-88a4-eb2649db9fbd service cinder] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:18.377408 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:18.378323 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:18.378747 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:18.379163 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] PATH_INFO: `/v3/users/3a98ba22d17c4abcaed30cd5eb32cb5e` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:18.380818 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a98ba22d17c4abcaed30cd5eb32cb5e)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:18.384260 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:18.724797 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a98ba22d17c4abcaed30cd5eb32cb5e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:18.725469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a98ba22d17c4abcaed30cd5eb32cb5e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:18.726063 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a98ba22d17c4abcaed30cd5eb32cb5e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:18.739230 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a98ba22d17c4abcaed30cd5eb32cb5e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:18.747224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a98ba22d17c4abcaed30cd5eb32cb5e'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:18.824349 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1452/2941] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:34:18 2022] DELETE /identity/v3/users/3a98ba22d17c4abcaed30cd5eb32cb5e => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:18.828845 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f2c4-9c80-47b3-96b6-3e5f299ddf5d None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:18.829341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f2c4-9c80-47b3-96b6-3e5f299ddf5d None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:18.829451 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f2c4-9c80-47b3-96b6-3e5f299ddf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:18.848638 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-ac70f2c4-9c80-47b3-96b6-3e5f299ddf5d None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:18.849870 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ac70f2c4-9c80-47b3-96b6-3e5f299ddf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:18.909092 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1490/2942] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 19:34:18.918656 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:18.920067 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:18.960531 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:18.978593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b32bba-2c98-4e5d-abef-5c4753646ffa admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:18.983310 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dd24-995d-4337-a6b7-5f9edfd8caca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:18.983914 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dd24-995d-4337-a6b7-5f9edfd8caca service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:18.984195 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dd24-995d-4337-a6b7-5f9edfd8caca service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:18.984469 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dd24-995d-4337-a6b7-5f9edfd8caca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:18.993690 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02dd24-995d-4337-a6b7-5f9edfd8caca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:18.995211 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02dd24-995d-4337-a6b7-5f9edfd8caca service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:19.004802 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1453/2943] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:19.425719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70f2c4-9c80-47b3-96b6-3e5f299ddf5d None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:19.427300 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-ac70f2c4-9c80-47b3-96b6-3e5f299ddf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:19.439747 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70f2c4-9c80-47b3-96b6-3e5f299ddf5d None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:19.450250 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:19.450907 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:19.451257 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:19.451544 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] PATH_INFO: `/v3/projects/9f520f7d1f554ccaaecca645b648129a` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:19.452692 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f520f7d1f554ccaaecca645b648129a)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:19.455419 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:19.568308 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:19.580660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f520f7d1f554ccaaecca645b648129a', 'request_id': 'req-54b5d509-d40f-41f1-8215-fdcf4995dc65'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:19.586806 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f520f7d1f554ccaaecca645b648129a', 'request_id': 'req-54b5d509-d40f-41f1-8215-fdcf4995dc65'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:19.670978 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f520f7d1f554ccaaecca645b648129a', 'request_id': 'req-54b5d509-d40f-41f1-8215-fdcf4995dc65'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:19.682055 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] The token cache is being invalidate because project 9f520f7d1f554ccaaecca645b648129a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:34:19.682350 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:19.683525 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:19.685019 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1491/2944] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:34:19 2022] DELETE /identity/v3/projects/9f520f7d1f554ccaaecca645b648129a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 19:34:19.688953 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dd24-995d-4337-a6b7-5f9edfd8caca service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:19.699192 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dd24-995d-4337-a6b7-5f9edfd8caca service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:19.760059 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:19.761382 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:19.761669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:19.761945 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] PATH_INFO: `/v3/users/15dfd997884e49e2b89481e3af6c2374` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:19.763111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15dfd997884e49e2b89481e3af6c2374)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:19.765037 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:19.957485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15dfd997884e49e2b89481e3af6c2374'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:19.958104 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15dfd997884e49e2b89481e3af6c2374'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:19.958720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15dfd997884e49e2b89481e3af6c2374'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:19.971095 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15dfd997884e49e2b89481e3af6c2374'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:19.978016 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-4558d976-38cf-430a-b60e-7f22a9b5cd8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15dfd997884e49e2b89481e3af6c2374'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:20.035088 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1454/2945] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:34:19 2022] DELETE /identity/v3/users/15dfd997884e49e2b89481e3af6c2374 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:20.427442 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:20.428949 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:20.442516 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5d509-d40f-41f1-8215-fdcf4995dc65 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:20.501103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:20.501745 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:20.502023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:20.502297 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] PATH_INFO: `/v3/projects/df6193c170c14c81b473fdbb1fc52bca` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:20.503456 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df6193c170c14c81b473fdbb1fc52bca)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:20.505395 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:20.629922 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df6193c170c14c81b473fdbb1fc52bca', 'request_id': 'req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:20.636253 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df6193c170c14c81b473fdbb1fc52bca', 'request_id': 'req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:20.710727 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df6193c170c14c81b473fdbb1fc52bca', 'request_id': 'req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:20.722240 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] The token cache is being invalidate because project df6193c170c14c81b473fdbb1fc52bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:34:20.722532 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:20.723219 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-60ec82cc-11fd-4f98-b31e-d00d1b8e28d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:34:20.724779 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1492/2946] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:34:20 2022] DELETE /identity/v3/projects/df6193c170c14c81b473fdbb1fc52bca => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:20.729705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d158d3-0734-40c4-9cfa-a4a4e9d2bda9 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:20.730173 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d158d3-0734-40c4-9cfa-a4a4e9d2bda9 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:20.734510 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d158d3-0734-40c4-9cfa-a4a4e9d2bda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:20.750676 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-65d158d3-0734-40c4-9cfa-a4a4e9d2bda9 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:20.751951 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-65d158d3-0734-40c4-9cfa-a4a4e9d2bda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:20.805437 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:20.815129 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1455/2947] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:20.822804 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628589fd-48f9-4b74-bc7e-4209c0eaa98a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:20.823094 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628589fd-48f9-4b74-bc7e-4209c0eaa98a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:20.823374 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628589fd-48f9-4b74-bc7e-4209c0eaa98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:20.845223 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-628589fd-48f9-4b74-bc7e-4209c0eaa98a None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:20.846434 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-628589fd-48f9-4b74-bc7e-4209c0eaa98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:20.854358 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1493/2948] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:20.858925 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d158d3-0734-40c4-9cfa-a4a4e9d2bda9 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:20.868960 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d158d3-0734-40c4-9cfa-a4a4e9d2bda9 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:20.873777 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2f878-8892-4665-b24a-ce22822ce887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:20.874375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2f878-8892-4665-b24a-ce22822ce887 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:20.874660 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2f878-8892-4665-b24a-ce22822ce887 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:20.874933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2f878-8892-4665-b24a-ce22822ce887 admin admin] PATH_INFO: `/v3/domains` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:20.875570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2f878-8892-4665-b24a-ce22822ce887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 19:34:20.875870 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2f878-8892-4665-b24a-ce22822ce887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:20.877501 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2f878-8892-4665-b24a-ce22822ce887 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:20.889571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1456/2949] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 8 19:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 19:34:20.894001 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628589fd-48f9-4b74-bc7e-4209c0eaa98a None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:20.904000 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628589fd-48f9-4b74-bc7e-4209c0eaa98a None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:20.908931 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6957f-e61e-487c-aa83-c879a14055e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:20.909565 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6957f-e61e-487c-aa83-c879a14055e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:20.909850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6957f-e61e-487c-aa83-c879a14055e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:20.910126 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6957f-e61e-487c-aa83-c879a14055e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:20.910755 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6957f-e61e-487c-aa83-c879a14055e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:20.912653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6957f-e61e-487c-aa83-c879a14055e8 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:20.998160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1494/2950] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 8 19:34:20 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 19:34:21.002552 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2f878-8892-4665-b24a-ce22822ce887 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:21.013330 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2f878-8892-4665-b24a-ce22822ce887 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:21.068380 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a9035-00cd-42fb-883c-4322c69d859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:21.068972 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a9035-00cd-42fb-883c-4322c69d859b admin admin] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:21.069278 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a9035-00cd-42fb-883c-4322c69d859b admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:21.069551 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a9035-00cd-42fb-883c-4322c69d859b admin admin] PATH_INFO: `/v3/users` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:21.070170 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a9035-00cd-42fb-883c-4322c69d859b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:21.072069 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a9035-00cd-42fb-883c-4322c69d859b admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:21.161529 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1457/2951] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 8 19:34:21 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 19:34:21.165788 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6957f-e61e-487c-aa83-c879a14055e8 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:21.177036 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6957f-e61e-487c-aa83-c879a14055e8 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:21.181715 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d53b0-ceb7-4b68-8975-41fbc78b9e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:21.182319 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d53b0-ceb7-4b68-8975-41fbc78b9e54 admin admin] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:21.182601 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d53b0-ceb7-4b68-8975-41fbc78b9e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:21.182881 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d53b0-ceb7-4b68-8975-41fbc78b9e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:21.184023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d53b0-ceb7-4b68-8975-41fbc78b9e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:21.189178 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d53b0-ceb7-4b68-8975-41fbc78b9e54 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:21.195886 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1495/2952] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:21.200726 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a9035-00cd-42fb-883c-4322c69d859b admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:21.210866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a9035-00cd-42fb-883c-4322c69d859b admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:21.215488 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf0ade-573d-4b37-b702-8e6e470dde59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:21.216083 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf0ade-573d-4b37-b702-8e6e470dde59 admin admin] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:21.216365 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf0ade-573d-4b37-b702-8e6e470dde59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:21.216642 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf0ade-573d-4b37-b702-8e6e470dde59 admin admin] PATH_INFO: `/v3/roles` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:21.217562 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf0ade-573d-4b37-b702-8e6e470dde59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:21.219180 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf0ade-573d-4b37-b702-8e6e470dde59 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:21.229832 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1458/2953] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 8 19:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 19:34:21.234161 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d53b0-ceb7-4b68-8975-41fbc78b9e54 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:21.244242 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d53b0-ceb7-4b68-8975-41fbc78b9e54 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:21.249274 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b975b-4006-4371-ac3c-fc58efc091e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:21.249916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b975b-4006-4371-ac3c-fc58efc091e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:21.250192 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b975b-4006-4371-ac3c-fc58efc091e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:21.250491 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b975b-4006-4371-ac3c-fc58efc091e0 admin admin] PATH_INFO: `/v3/projects/86025b26a5064527805e36c43d11e12c/users/888890c77a7f47b6b30cdbcef13e7d54/roles/eccc3f8721b1466b8a1968b77e5c464f` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:21.258850 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:21.265856 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b975b-4006-4371-ac3c-fc58efc091e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86025b26a5064527805e36c43d11e12c, user_id=888890c77a7f47b6b30cdbcef13e7d54, role_id=eccc3f8721b1466b8a1968b77e5c464f)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:21.268518 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b975b-4006-4371-ac3c-fc58efc091e0 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:21.337590 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1496/2954] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 8 19:34:21 2022] PUT /identity/v3/projects/86025b26a5064527805e36c43d11e12c/users/888890c77a7f47b6b30cdbcef13e7d54/roles/eccc3f8721b1466b8a1968b77e5c464f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:34:21.342840 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00e98e-4213-4deb-9fa7-95bfa9165693 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:21.343126 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00e98e-4213-4deb-9fa7-95bfa9165693 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:21.343448 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00e98e-4213-4deb-9fa7-95bfa9165693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:21.363263 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-1b00e98e-4213-4deb-9fa7-95bfa9165693 None None] MFA Rules not processed for user `ba1856b68bd244e69db7d867dbb38da3`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:21.364518 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-1b00e98e-4213-4deb-9fa7-95bfa9165693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:21.425283 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1459/2955] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:21.435561 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b975b-4006-4371-ac3c-fc58efc091e0 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:21.437341 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-f63b975b-4006-4371-ac3c-fc58efc091e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:21.494797 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b975b-4006-4371-ac3c-fc58efc091e0 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:21.499296 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70078-2829-4423-b100-9ea1bc33bbd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:21.499903 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70078-2829-4423-b100-9ea1bc33bbd5 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:21.500188 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70078-2829-4423-b100-9ea1bc33bbd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:21.500461 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af70078-2829-4423-b100-9ea1bc33bbd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:21.509653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af70078-2829-4423-b100-9ea1bc33bbd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:21.511011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af70078-2829-4423-b100-9ea1bc33bbd5 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:21.520576 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1497/2956] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:22.714691 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b035969-a567-49a7-aa88-647c34a4c0ee None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:22.715449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b035969-a567-49a7-aa88-647c34a4c0ee None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:22.715449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b035969-a567-49a7-aa88-647c34a4c0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:22.744436 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-6b035969-a567-49a7-aa88-647c34a4c0ee None None] MFA Rules not processed for user `888890c77a7f47b6b30cdbcef13e7d54`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:22.745705 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-6b035969-a567-49a7-aa88-647c34a4c0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:22.805908 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:22.817613 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1460/2957] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:22.836062 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70078-2829-4423-b100-9ea1bc33bbd5 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:22.837575 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-8af70078-2829-4423-b100-9ea1bc33bbd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:22.896873 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af70078-2829-4423-b100-9ea1bc33bbd5 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:22.900453 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53660bd-bebb-4752-9736-613a8eb6050a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605b5d4c0fc9409a94dad22c77c67332', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:22.901065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53660bd-bebb-4752-9736-613a8eb6050a service cinder] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:22.901370 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53660bd-bebb-4752-9736-613a8eb6050a service cinder] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:22.901645 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53660bd-bebb-4752-9736-613a8eb6050a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:22.910485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53660bd-bebb-4752-9736-613a8eb6050a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:22.911915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53660bd-bebb-4752-9736-613a8eb6050a service cinder] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:22.921914 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1498/2958] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:24.775932 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10dd59-d3a4-471b-924e-cfa0ebc4f38c None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:24.776423 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10dd59-d3a4-471b-924e-cfa0ebc4f38c None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:24.776511 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10dd59-d3a4-471b-924e-cfa0ebc4f38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:24.796776 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ac10dd59-d3a4-471b-924e-cfa0ebc4f38c None None] MFA Rules not processed for user `888890c77a7f47b6b30cdbcef13e7d54`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:24.798070 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ac10dd59-d3a4-471b-924e-cfa0ebc4f38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:24.805085 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1461/2959] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 19:34:25.036855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53660bd-bebb-4752-9736-613a8eb6050a service cinder] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:34:25.038398 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-a53660bd-bebb-4752-9736-613a8eb6050a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:25.050427 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53660bd-bebb-4752-9736-613a8eb6050a service cinder] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:34:25.053923 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e82b9c-96c2-4479-8cc0-815d4322d6f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:34:25.054510 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e82b9c-96c2-4479-8cc0-815d4322d6f2 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:25.054785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e82b9c-96c2-4479-8cc0-815d4322d6f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:25.055054 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e82b9c-96c2-4479-8cc0-815d4322d6f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:25.056691 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-88e82b9c-96c2-4479-8cc0-815d4322d6f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:25.067031 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e82b9c-96c2-4479-8cc0-815d4322d6f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:34:25.068471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e82b9c-96c2-4479-8cc0-815d4322d6f2 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:34:25.077499 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1499/2960] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:34:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:34:34.791100 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:35.072848 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:34:42.239217 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c17657-dff0-467b-afd2-9dfb293f2490 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:34:42.239751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c17657-dff0-467b-afd2-9dfb293f2490 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:34:42.239751 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c17657-dff0-467b-afd2-9dfb293f2490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:34:42.258596 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-b0c17657-dff0-467b-afd2-9dfb293f2490 None None] MFA Rules not processed for user `888890c77a7f47b6b30cdbcef13e7d54`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:34:42.259815 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b0c17657-dff0-467b-afd2-9dfb293f2490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:34:42.267208 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1462/2961] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:34:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:34:52.253844 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:35:03.671826 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c8f7-509f-4851-8a33-3210334d2d75 None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:35:03.672371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c8f7-509f-4851-8a33-3210334d2d75 None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:35:03.672371 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c8f7-509f-4851-8a33-3210334d2d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:35:03.733437 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-1665c8f7-509f-4851-8a33-3210334d2d75 None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:35:03.734657 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-1665c8f7-509f-4851-8a33-3210334d2d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:35:03.807660 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1500/2962] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 8 19:35:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:35:03.943783 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c17657-dff0-467b-afd2-9dfb293f2490 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:35:03.945229 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-b0c17657-dff0-467b-afd2-9dfb293f2490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:35:04.034887 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c17657-dff0-467b-afd2-9dfb293f2490 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:35:04.040159 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f1dad-6398-4d7e-8dc2-cdeb19786df0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:35:04.040757 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f1dad-6398-4d7e-8dc2-cdeb19786df0 service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:35:04.041037 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f1dad-6398-4d7e-8dc2-cdeb19786df0 service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:35:04.041337 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f1dad-6398-4d7e-8dc2-cdeb19786df0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:35:04.049300 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f1dad-6398-4d7e-8dc2-cdeb19786df0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:35:04.050866 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f1dad-6398-4d7e-8dc2-cdeb19786df0 service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:35:04.059032 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1463/2963] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 19:35:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 19:35:13.804744 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=47 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:35:14.055762 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=41 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:35:22.016769 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d156a3-53e0-4c60-9053-bf3fd6db494a None None] REQUEST_METHOD: `POST` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:35:22.017239 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d156a3-53e0-4c60-9053-bf3fd6db494a None None] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:35:22.017387 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d156a3-53e0-4c60-9053-bf3fd6db494a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:35:22.037520 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.auth.core [None req-b9d156a3-53e0-4c60-9053-bf3fd6db494a None None] MFA Rules not processed for user `3038d1d505654ffb95b78fbc9005baf4`. Rule list: `[]` (Enabled: `True`). {{(pid=88173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:35:22.038789 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b9d156a3-53e0-4c60-9053-bf3fd6db494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:35:22.045828 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1501/2964] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 8 19:35:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:35:22.058004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f1dad-6398-4d7e-8dc2-cdeb19786df0 service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:35:22.069137 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f1dad-6398-4d7e-8dc2-cdeb19786df0 service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:35:22.074435 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd5fdc-4979-4ed4-ab08-01d7397660ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88a9b2cee21f4831b221d113ddbbd29d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:35:22.075036 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd5fdc-4979-4ed4-ab08-01d7397660ec service swift] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:35:22.075389 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd5fdc-4979-4ed4-ab08-01d7397660ec service swift] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:35:22.075669 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd5fdc-4979-4ed4-ab08-01d7397660ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:35:22.083712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd5fdc-4979-4ed4-ab08-01d7397660ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:35:22.085520 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd5fdc-4979-4ed4-ab08-01d7397660ec service swift] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:35:22.093536 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1464/2965] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 8 19:35:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 19:35:24.883959 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d156a3-53e0-4c60-9053-bf3fd6db494a None None] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:35:24.886450 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-b9d156a3-53e0-4c60-9053-bf3fd6db494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:35:24.956356 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d156a3-53e0-4c60-9053-bf3fd6db494a None None] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:35:24.961352 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f20f8d-1011-488c-aa60-ae8e8a167598 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:35:24.961957 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f20f8d-1011-488c-aa60-ae8e8a167598 service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:35:24.962234 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f20f8d-1011-488c-aa60-ae8e8a167598 service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:35:24.962504 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f20f8d-1011-488c-aa60-ae8e8a167598 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:35:24.964225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-70f20f8d-1011-488c-aa60-ae8e8a167598 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:35:24.975821 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f20f8d-1011-488c-aa60-ae8e8a167598 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:35:24.977934 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f20f8d-1011-488c-aa60-ae8e8a167598 service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:35:24.987851 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1502/2966] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:35:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:35:25.011689 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd5fdc-4979-4ed4-ab08-01d7397660ec service swift] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:35:25.027322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd5fdc-4979-4ed4-ab08-01d7397660ec service swift] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:35:25.033582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:35:25.034224 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:35:25.034509 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:35:25.034789 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] PATH_INFO: `/v3/users/888890c77a7f47b6b30cdbcef13e7d54` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:35:25.036387 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=888890c77a7f47b6b30cdbcef13e7d54)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:35:25.038358 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:35:25.212593 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '888890c77a7f47b6b30cdbcef13e7d54'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:35:25.213234 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '888890c77a7f47b6b30cdbcef13e7d54'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:35:25.214113 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '888890c77a7f47b6b30cdbcef13e7d54'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:35:25.228772 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '888890c77a7f47b6b30cdbcef13e7d54'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:35:25.236970 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-d45e67ad-5f4e-4d72-89ea-240c134d3b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '888890c77a7f47b6b30cdbcef13e7d54'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:35:25.329411 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1465/2967] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:35:25 2022] DELETE /identity/v3/users/888890c77a7f47b6b30cdbcef13e7d54 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:35:25.649761 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f20f8d-1011-488c-aa60-ae8e8a167598 service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:35:25.664009 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f20f8d-1011-488c-aa60-ae8e8a167598 service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:35:25.737540 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:35:25.738179 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:35:25.738459 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:35:25.738777 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] PATH_INFO: `/v3/projects/86025b26a5064527805e36c43d11e12c` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:35:25.739956 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=86025b26a5064527805e36c43d11e12c)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:35:25.742539 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:35:25.848313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86025b26a5064527805e36c43d11e12c', 'request_id': 'req-04c1949a-ad74-4e92-ab7f-f51b224269ff'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:35:25.855608 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86025b26a5064527805e36c43d11e12c', 'request_id': 'req-04c1949a-ad74-4e92-ab7f-f51b224269ff'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:35:25.926633 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86025b26a5064527805e36c43d11e12c', 'request_id': 'req-04c1949a-ad74-4e92-ab7f-f51b224269ff'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:35:25.945628 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] The token cache is being invalidate because project 86025b26a5064527805e36c43d11e12c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:35:25.946160 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:35:25.947259 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:35:25.949109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1503/2968] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:35:25 2022] DELETE /identity/v3/projects/86025b26a5064527805e36c43d11e12c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:35:35.324933 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:35:35.940281 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:35:41.839731 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ac5e3-f8a2-4818-afd7-bf36f74b33c6 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:35:41.840305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ac5e3-f8a2-4818-afd7-bf36f74b33c6 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:35:41.840305 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ac5e3-f8a2-4818-afd7-bf36f74b33c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:35:41.861764 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-ce1ac5e3-f8a2-4818-afd7-bf36f74b33c6 None None] MFA Rules not processed for user `f9ce84e784a246e4a7e79695a1f81365`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:35:41.862889 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-ce1ac5e3-f8a2-4818-afd7-bf36f74b33c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:35:41.918331 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1466/2969] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:35:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:35:51.911183 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=25 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:36:12.202183 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:36:12.203670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:36:12.286864 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1949a-ad74-4e92-ab7f-f51b224269ff admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:36:12.291987 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7feed-7ad7-4c5b-bf2c-1b91a8982d46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:36:12.292623 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7feed-7ad7-4c5b-bf2c-1b91a8982d46 service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:36:12.292909 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7feed-7ad7-4c5b-bf2c-1b91a8982d46 service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:36:12.293221 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7feed-7ad7-4c5b-bf2c-1b91a8982d46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:36:12.294933 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-56a7feed-7ad7-4c5b-bf2c-1b91a8982d46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:36:12.306484 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7feed-7ad7-4c5b-bf2c-1b91a8982d46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:36:12.308075 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7feed-7ad7-4c5b-bf2c-1b91a8982d46 service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:36:12.318218 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1504/2970] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:36:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:36:22.313868 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=37 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:37:52.423054 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a363a-6bb9-4a57-8a70-0c67dac8e0e7 None None] REQUEST_METHOD: `POST` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:37:52.423570 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a363a-6bb9-4a57-8a70-0c67dac8e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:37:52.423675 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a363a-6bb9-4a57-8a70-0c67dac8e0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:37:52.443537 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.auth.core [None req-351a363a-6bb9-4a57-8a70-0c67dac8e0e7 None None] MFA Rules not processed for user `f9ce84e784a246e4a7e79695a1f81365`. Rule list: `[]` (Enabled: `True`). {{(pid=88172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 19:37:52.444778 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-351a363a-6bb9-4a57-8a70-0c67dac8e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:37:52.451619 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1467/2971] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 8 19:37:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 19:38:00.510599 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7feed-7ad7-4c5b-bf2c-1b91a8982d46 service placement] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:38:00.512105 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-56a7feed-7ad7-4c5b-bf2c-1b91a8982d46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:38:00.605258 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7feed-7ad7-4c5b-bf2c-1b91a8982d46 service placement] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:38:00.610702 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a78f8-03f0-4047-9c82-209d4fab6c6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:38:00.611304 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a78f8-03f0-4047-9c82-209d4fab6c6a service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:38:00.611584 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a78f8-03f0-4047-9c82-209d4fab6c6a service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:38:00.611855 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a78f8-03f0-4047-9c82-209d4fab6c6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:38:00.613596 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-711a78f8-03f0-4047-9c82-209d4fab6c6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:38:00.623997 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a78f8-03f0-4047-9c82-209d4fab6c6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:38:00.625577 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a78f8-03f0-4047-9c82-209d4fab6c6a service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:38:00.634938 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1505/2972] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:38:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:38:02.438478 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=3 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:38:10.630916 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=43 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:39:18.883004 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a363a-6bb9-4a57-8a70-0c67dac8e0e7 None None] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:39:18.884524 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-351a363a-6bb9-4a57-8a70-0c67dac8e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:39:18.942037 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a363a-6bb9-4a57-8a70-0c67dac8e0e7 None None] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:39:18.945592 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eed2d-64e5-4129-b307-ca78f01a4a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:39:18.946199 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eed2d-64e5-4129-b307-ca78f01a4a33 service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:39:18.946480 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eed2d-64e5-4129-b307-ca78f01a4a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:39:18.946756 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eed2d-64e5-4129-b307-ca78f01a4a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:39:18.948489 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-962eed2d-64e5-4129-b307-ca78f01a4a33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:39:18.958207 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962eed2d-64e5-4129-b307-ca78f01a4a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:39:18.959636 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962eed2d-64e5-4129-b307-ca78f01a4a33 service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:39:18.968080 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1468/2973] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:39:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:39:28.964094 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=26 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:39:44.449685 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a78f8-03f0-4047-9c82-209d4fab6c6a service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:39:44.454350 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-711a78f8-03f0-4047-9c82-209d4fab6c6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:39:44.465391 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a78f8-03f0-4047-9c82-209d4fab6c6a service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:39:44.471053 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13498bff-ec9f-4b30-8bb6-c94ef76c4fe9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:39:44.477136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498bff-ec9f-4b30-8bb6-c94ef76c4fe9 service nova] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:39:44.477136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498bff-ec9f-4b30-8bb6-c94ef76c4fe9 service nova] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:39:44.477136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13498bff-ec9f-4b30-8bb6-c94ef76c4fe9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:39:44.477136 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-13498bff-ec9f-4b30-8bb6-c94ef76c4fe9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:39:44.487350 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13498bff-ec9f-4b30-8bb6-c94ef76c4fe9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:39:44.488927 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13498bff-ec9f-4b30-8bb6-c94ef76c4fe9 service nova] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:39:44.501211 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1506/2974] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:39:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:39:44.819845 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eed2d-64e5-4129-b307-ca78f01a4a33 service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:39:44.833111 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eed2d-64e5-4129-b307-ca78f01a4a33 service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:39:44.838190 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fa835-7745-4d63-8e88-fbb60616c41f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:39:44.839179 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fa835-7745-4d63-8e88-fbb60616c41f service neutron] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:39:44.839463 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fa835-7745-4d63-8e88-fbb60616c41f service neutron] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:39:44.839817 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fa835-7745-4d63-8e88-fbb60616c41f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:39:44.841712 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-367fa835-7745-4d63-8e88-fbb60616c41f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:39:44.856221 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fa835-7745-4d63-8e88-fbb60616c41f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:39:44.857721 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fa835-7745-4d63-8e88-fbb60616c41f service neutron] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:39:44.869285 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1469/2975] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:39:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:39:46.079589 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13498bff-ec9f-4b30-8bb6-c94ef76c4fe9 service nova] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:39:46.089023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13498bff-ec9f-4b30-8bb6-c94ef76c4fe9 service nova] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:39:46.092693 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edbc13-eabf-4732-9aea-bc908fde10f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:39:46.093676 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edbc13-eabf-4732-9aea-bc908fde10f0 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:39:46.093958 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edbc13-eabf-4732-9aea-bc908fde10f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:39:46.094228 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edbc13-eabf-4732-9aea-bc908fde10f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:39:46.096508 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-66edbc13-eabf-4732-9aea-bc908fde10f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:39:46.112659 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edbc13-eabf-4732-9aea-bc908fde10f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:39:46.114128 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edbc13-eabf-4732-9aea-bc908fde10f0 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:39:46.125890 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1507/2976] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:39:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:39:54.864118 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=6 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:39:55.416586 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fa835-7745-4d63-8e88-fbb60616c41f service neutron] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:39:55.430311 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fa835-7745-4d63-8e88-fbb60616c41f service neutron] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:39:55.434264 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c737ca-38b7-4d09-be45-0c78d3dd04ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b29c5745a8e42ff8232e5ca22b4a4ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:39:55.434936 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c737ca-38b7-4d09-be45-0c78d3dd04ed service nova] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:39:55.435220 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c737ca-38b7-4d09-be45-0c78d3dd04ed service nova] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:39:55.435493 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c737ca-38b7-4d09-be45-0c78d3dd04ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:39:55.437401 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-60c737ca-38b7-4d09-be45-0c78d3dd04ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:39:55.450406 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c737ca-38b7-4d09-be45-0c78d3dd04ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:39:55.452009 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c737ca-38b7-4d09-be45-0c78d3dd04ed service nova] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:39:55.463451 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1470/2977] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:39:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:39:56.120360 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=12 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:39:59.121554 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edbc13-eabf-4732-9aea-bc908fde10f0 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:39:59.130801 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edbc13-eabf-4732-9aea-bc908fde10f0 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:39:59.134441 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b35bf1-51c8-434c-afe4-efbe1a8e18c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcf30ef695154d3a97d7e589aca21233', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:39:59.135034 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35bf1-51c8-434c-afe4-efbe1a8e18c9 service neutron] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:39:59.135313 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35bf1-51c8-434c-afe4-efbe1a8e18c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:39:59.135592 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35bf1-51c8-434c-afe4-efbe1a8e18c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:39:59.137338 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d0b35bf1-51c8-434c-afe4-efbe1a8e18c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:39:59.201023 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b35bf1-51c8-434c-afe4-efbe1a8e18c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:39:59.202514 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b35bf1-51c8-434c-afe4-efbe1a8e18c9 service neutron] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:39:59.212496 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1508/2978] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:40:02.506322 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c737ca-38b7-4d09-be45-0c78d3dd04ed service nova] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:40:02.518135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c737ca-38b7-4d09-be45-0c78d3dd04ed service nova] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:40:02.523375 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8804296-a571-4042-b36d-8c286770bb1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:40:02.523980 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8804296-a571-4042-b36d-8c286770bb1f service placement] REQUEST_METHOD: `GET` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:40:02.524262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8804296-a571-4042-b36d-8c286770bb1f service placement] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:40:02.528720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8804296-a571-4042-b36d-8c286770bb1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:40:02.528720 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-c8804296-a571-4042-b36d-8c286770bb1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:40:02.538633 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8804296-a571-4042-b36d-8c286770bb1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:40:02.540216 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8804296-a571-4042-b36d-8c286770bb1f service placement] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:40:02.551519 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1471/2979] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:40:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:40:06.288316 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b35bf1-51c8-434c-afe4-efbe1a8e18c9 service neutron] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:40:06.289915 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d0b35bf1-51c8-434c-afe4-efbe1a8e18c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:40:06.303103 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b35bf1-51c8-434c-afe4-efbe1a8e18c9 service neutron] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:40:06.308040 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:40:06.308705 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:40:06.308989 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:40:06.309324 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] PATH_INFO: `/v3/users/f9ce84e784a246e4a7e79695a1f81365` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:40:06.310485 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ce84e784a246e4a7e79695a1f81365)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:40:06.312408 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:40:06.492688 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ce84e784a246e4a7e79695a1f81365'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:06.493392 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ce84e784a246e4a7e79695a1f81365'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:06.494059 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ce84e784a246e4a7e79695a1f81365'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:06.501670 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ce84e784a246e4a7e79695a1f81365'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:06.522038 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ce84e784a246e4a7e79695a1f81365'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:06.595011 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1509/2980] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:40:06 2022] DELETE /identity/v3/users/f9ce84e784a246e4a7e79695a1f81365 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:40:06.840135 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8804296-a571-4042-b36d-8c286770bb1f service placement] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:40:06.863427 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8804296-a571-4042-b36d-8c286770bb1f service placement] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:40:06.921965 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:40:06.922692 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:40:06.922971 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:40:06.923244 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] PATH_INFO: `/v3/projects/8a79f3fe2a954504a155388b7780651d` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:40:06.924395 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a79f3fe2a954504a155388b7780651d)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:40:06.926345 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:40:07.021885 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a79f3fe2a954504a155388b7780651d', 'request_id': 'req-f04c1928-1b41-4199-a851-0269b1411171'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.028014 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a79f3fe2a954504a155388b7780651d', 'request_id': 'req-f04c1928-1b41-4199-a851-0269b1411171'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.033648 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a79f3fe2a954504a155388b7780651d', 'request_id': 'req-f04c1928-1b41-4199-a851-0269b1411171'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.088464 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] The token cache is being invalidate because project 8a79f3fe2a954504a155388b7780651d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:40:07.088753 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.089428 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.098485 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1472/2981] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:40:06 2022] DELETE /identity/v3/projects/8a79f3fe2a954504a155388b7780651d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:40:07.098625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:40:07.098625 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:40:07.110085 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964eca1-51ba-4e9b-959e-33063422d2d1 admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:40:07.167108 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:40:07.167653 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:40:07.167929 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:40:07.168200 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] PATH_INFO: `/v3/users/247116c6453941b58cfbf49edda03b01` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:40:07.169340 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] RBAC: Authorizing `identity:delete_user(user_id=247116c6453941b58cfbf49edda03b01)` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:40:07.171225 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:40:07.176644 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:40:07.369065 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '247116c6453941b58cfbf49edda03b01'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.369859 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '247116c6453941b58cfbf49edda03b01'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.370471 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '247116c6453941b58cfbf49edda03b01'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.377574 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '247116c6453941b58cfbf49edda03b01'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.390813 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.notifications [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '247116c6453941b58cfbf49edda03b01'} {{(pid=88173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.475836 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1510/2982] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 8 19:40:07 2022] DELETE /identity/v3/users/247116c6453941b58cfbf49edda03b01 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:40:07.702688 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] Authenticating user token {{(pid=88172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:40:07.705262 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.fernet_utils [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:40:07.717786 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c1928-1b41-4199-a851-0269b1411171 admin admin] Validating token access rules against request {{(pid=88172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:40:07.779378 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba1856b68bd244e69db7d867dbb38da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a0304ec85d64b64aa7cbc3f8a24053a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:40:07.780019 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:40:07.780297 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] SCRIPT_NAME: `/identity` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:40:07.780571 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] PATH_INFO: `/v3/projects/3a99610fdc2b483784e420adb57ba5b9` {{(pid=88172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:40:07.781725 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a99610fdc2b483784e420adb57ba5b9)` {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:40:07.783643 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] RBAC: Authorization granted {{(pid=88172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:40:07.790909 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:40:07.890333 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a99610fdc2b483784e420adb57ba5b9', 'request_id': 'req-3b513d71-2574-454b-a8e3-ebc01a52ecca'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.896958 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a99610fdc2b483784e420adb57ba5b9', 'request_id': 'req-3b513d71-2574-454b-a8e3-ebc01a52ecca'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.902456 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a99610fdc2b483784e420adb57ba5b9', 'request_id': 'req-3b513d71-2574-454b-a8e3-ebc01a52ecca'} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.980223 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] The token cache is being invalidate because project 3a99610fdc2b483784e420adb57ba5b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 19:40:07.980449 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.981092 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG keystone.notifications [None req-3b513d71-2574-454b-a8e3-ebc01a52ecca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 19:40:07.982686 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: [pid: 88172|app: 0|req: 1473/2983] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 8 19:40:07 2022] DELETE /identity/v3/projects/3a99610fdc2b483784e420adb57ba5b9 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 19:40:17.471462 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:40:18.038582 dv-cinder-856463-3-iscsi devstack@keystone.service[88172]: DEBUG dbcounter [-] [88172] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 19:41:15.138681 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] Authenticating user token {{(pid=88173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 19:41:15.140097 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:41:15.241430 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fccece-03bc-4b88-b3a2-89a8fd09aefc admin admin] Validating token access rules against request {{(pid=88173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 19:41:15.307895 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09488770-546b-4551-8110-73ed22f82dc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a074714e365648b88c9d3a0495a17b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '371cca2196e34bd181a09086679cb0e7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 19:41:15.308509 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09488770-546b-4551-8110-73ed22f82dc7 service placement] REQUEST_METHOD: `GET` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 19:41:15.308785 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09488770-546b-4551-8110-73ed22f82dc7 service placement] SCRIPT_NAME: `/identity` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 19:41:15.309109 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09488770-546b-4551-8110-73ed22f82dc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 19:41:15.310849 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.fernet_utils [None req-09488770-546b-4551-8110-73ed22f82dc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 19:41:15.322131 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09488770-546b-4551-8110-73ed22f82dc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 19:41:15.323719 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09488770-546b-4551-8110-73ed22f82dc7 service placement] RBAC: Authorization granted {{(pid=88173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 19:41:15.333522 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: [pid: 88173|app: 0|req: 1511/2984] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 8 19:41:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 19:41:25.329296 dv-cinder-856463-3-iscsi devstack@keystone.service[88173]: DEBUG dbcounter [-] [88173] Writing DB stats keystone:SELECT=71 {{(pid=88173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}